He cling to the Gedo Mazo for life didn't he?I think that people believe that summoning the Gedou Mazou was the cause of him becoming unstable and malnourished which is highly inaccurate.
I think it's because Rinnegan techniques take a massive toll on his life force, or chakra.