I have a few theories about that...
First of all, Jiraya was probably the only one who could pursue Orochimaru effectively and thus keep an eye out for potential trouble for Konoha. In addition to that, I doubt Jiraya had given up saving his old friend in some way up until the point where he killed Hiruzen
Second of all, let's face it Jiraya is a hopeless pervert, I think he knows as well as we do that he wouldn't make a very good parent. Yes, we all agree that having SOME parent is better than growing up alone and hated, but Jiraya wasn't there, he can't have known what's going on with Naruto. He trusted in Hiruzen to ensure a happy childhood for the boy, a happier one than the one he thought he'd inflict on him
Third of all, and I suppose this is no small factor, but very underestimated by people: We have the psychology factor. Jiraya loved Minato and believed that he is the Chosen One, the Savior of the World. He didn't love him just as a teacher, he loved him as a father, as he mentions a few times (even seeing himself as a grandfather-figure for Naruto later on) and later as a friend. After what happened with Nagato, Yahiko, Konan, Orochimaru and Tsunade, losing his only other beloved student/friend must have shaken him. Naruto being a little carbon-copy of his father can't have helped the case either. Perhaps Jiraya couldn't, for the time being, face his student's son as it just brought too many painful memories. That, combined with point 2 and 1, might be a big enough reason to force Jiraya to leave.
