Even if you say this, Madara was still not dead as he was alive after Kaguya was sealed. But regardless, Kaguya would have still had his chakra along with everyone else's chakra she was getting from the tree. The balls disintegrated after Kaguya went into the lava dimension.
Travelling into another dimension would cause the presence of Naruto's chakra in the previous dimension to disappear. He would no longer be in that dimension. His chakra wouldn't just hang around without him. The deactivation of his cloak doesn't change his chakra still being there, thus it has no effect on the gudoudama.
In the case of Madara and Kamui, he was bisected before warping, and only half his body made it through. That means his other half was still in the real world. And as it's been shown in this manga, even a single cell still carries the person's chakra (Hashirama) so half of Madara's body would be enough to keep the gudoudama solid.
Kaguya was absorbing chakra from everyone. The half of Madara's body that was left behind by her was most likely devoid of his chakra, causing the gudoudama to disintegrate after Kaguya, who now possessed Madara's chakra, left to another dimension.