[boolean logic]
Everything fits beautifully.
(unless people know about someone with red hair -and extreme vitality- that is not uzumaki, and someone that is not hyuuga and was naturally born with byakugan).
Assuming that all of the pieces on the field come from Kaguya.
Allow me to posit that the Ototsuki are space aliens ("Saiyans"). They have the Byakugan and have honed the ability to use their body's chakra.
So, one of them happens to crash land, or something, onto the planet. Perhaps she's some refugee from a celestial war, perhaps she's an escaped convict - either way, it becomes apparent that she's quite the martial bad-ass compared to your mk1 human. But there's just so much fighting going on, and if she only had more power.
Well, a little bird tells her that there's this 'god tree' thing. Perhaps she should go check it out - gods have power, and humans have used trees for a long damned time to get what they want. The logic doesn't have to be reduced to a truth table.
When Kaguya-Hime arrives on the island, she's greeted, perhaps with some hostility, by the tree's tenants. Apparently, they've been dabbling with chakra usage in some relatively creative ways - rituals, chants, objects and symbols that react to chakra and produce a wide range of effects (though most seem to be concerned with sealing chakra away... probably because they are tending a tree that gets pissed when they so much as take a leaf the wrong way).
So, Kaguya eats the fruit and rapes one of the shamans (or perhaps it was voluntary on his part - though I suspect a being of her power could make it quite involuntary). Normal people have children because they are tired of mowing the lawn. Dictators have kids because they are tired of mowing the peasants, and by Golly, Kaguya was getting tired of that chore.
Anyway - that pretty much sums up our lineages. The Sage's brother is a free floating factor that can't be accounted for until more is known about his existence. We could postulate that the Uzumaki and Hyuuga descend from the Sage's brother - but it is just as (if not more) plausible that the Uzumaki and Hyuuga draw their lineages from outside of Kaguya and The Sage.
The Uzumaki were known for sealing jutsu that differed substantially from scrolls and symbols. The Hyuuga have yet to be depicted utilizing a ninjutsu. Even a simple clone or substitution jutsu that would be vital for closing the distance and/or evading and overpowering attack are not incorporated into their fighting style in the slightest.
I would make the argument that the Hyuuga did not receive chakra from the Sage, and cannot (or will not) use it for the purposes of ninjutsu.
Perhaps the Sage's brother was the real Sage; not lingering beyond his time, not tampering in worldly affairs, leaving little trace of his existence, etc.... basically - everything the heretic Hagoromo is not.
Of course - that's also operating off of literally no information other than: "The Sage of Six Paths had a brother." Which is about all we know of him.