I just like the fact that he actually had quite the hard life as well. He was chosen by the Toad Sage to be 'the one' or ambiguously special anyway and that was a lie, he never had any luck with Tsunade, he fought through many wars, couldn't stop or save his teammate, had his sensei and favourite student die and then was killed by his own pupils...and he still did it all with a smile on his face.
Not all tortured characters have to be stoic and silent. That's why I like Jiraiya anyway, and whilst i'm not crazy over him- he picked the perfect time in the manga to kick it.