Kaguya > Madara
Hagoromo > Madara
Kaguya had the power of the Chakra Fruit, which was significantly superior to that of Hagoromo's own, who was the Juubi jinchuuriki. We can conclude this based on what he said, and the way in which he said it ("Yeah, she was strong... Stronger than anyone else...").
Secondly, Hagoromo is still superior to Madara. By a fair bit too.
To begin with, consider that Naruto and Sasuke, the only ones capable of even challenging Madara, can only do so because Hagoromo granted each one half of his power. Sasuke received his Yin (spiritual) energy, and Naruto his Yang (physical) energy. In essence, Hagoromo is, in a sense, there himself, challenging Madara.
Next, Hagoromo, even before becoming the Juubi jinchuuriki, was insanely powerful. He and his brother together defeated the Juubi. Considering the fact that he and his brother were portrayed as equals, Hagoromo was at least as powerful as 50% of the Juubi prior to becoming its jinchuuriki. Also, the fact that he was able to perform Chibaku Tensei at the scale of creating the Moon on his deathbed is nothing short of spectacular. Nagato wasn't able to even keep Kurama contained with his Chibaku Tensei, and Hagoromo created the Moon with his, without the Juubi, while he was at his weakest (on his deathbed).
For those who argue that Madara has the Juubi's Eye, and Hagoromo didn't, please read the manga. Madara stated that the way to obtain the Juubi's Eye is for a Rinnegan user to approach the Moon. Hagoromo is a Rinnegan user, so he can pretty much go up to the Moon any time he likes and obtain the Juubi's Eye. Hell, he gave the Juubi's Eye to Sasuke, so he probably already had that power anyway.
And, for those who claim superiority for Madara based on the fact that he has the Shinju, it is just laughable. Hagoromo had the Shinju as well. Hell, so did Obito. After all, Obito used it on the Shinobi Alliance. The only way to do that is if you have that power in the first place. Hagoromo had the Shinju sealed in himself as well. If you claim he didn't, then pray tell, where on Earth did the Shinju go? Did it just vanish? No, of course not. Hagoromo had the Shinju sealed inside him as well.
Hagoromo was the jinchuuriki of the full Juubi, and that Juubi contained the power of the 9 bijuu, and then some ("the whole is greater than the sum of its parts"), not 8.5 bijuu, like Madara has. Madara is missing Yin Kurama. Kurama is the most powerful of the 9 bijuu, so his power is nothing to scoff at. Yang Kurama alone was powerful enough to take on several bijuu. This is, no matter what anyone thinks, a pretty big chunk of power.
In concusion, Hagoromo > Madara. Kaguya, stated as Hagoromo to be the strongest, is also stronger than Madara.
