Then he would have left them there and not bothered to call them back Because he choose Sasuke as a successor.
