I don't think he's hated. He's underestimated because we haven't seen any feats from him. His one fight was back in Part I (so the power scales were lower), and he had to use the RDS. He wasn't even powerful enough to pull out Orochimaru's soul, having to settle for just his arms.
We've never seen Prime Hiruzen, so many people underestimate him. Despite his hype, he hasn't provided any feats. Hashirama beat Madara and Kurama and Minato beat Obito and Kurama. We don't have any feats from Tobirama either, but, let's face it, he's a badass.
Oh, and he beat Izuna, who had the MS, and was supposedly about on par with MS Madara.