He was personally trained by one of the Sannin, and the other two of the legendary three were taken already by the other members of Team 7.
Also, just think of the moves you'd have to have to avoid being molested by multiple creepers at every turn.
No, he would not be as strong if he had remained in the village.