But yeah then it defuncts the concept that the Sage traveled around teaching people lol. But if he is the ancestor of all shinobi then everyone would either be part of the senju or uchiha clan.....which isn't true. not everyone has the sharingan or super strong body lol...So logically there must be another explanation.
The Sage did travel around teaching people. But probably not Ninjutsu. Most likely, he traveled around teaching his quasi-religious message of peace, and only taught Ninjutsu to his direct descendants. Eventually, as history blurred the facts about his life and the origin of chakra, the myth became that he went around teaching everyone ninjutsu.
Also, that's illogical. The Elder Son had a Dojutsu that was entirely different from anything else we've seen. Yet the Uchiha clan doesn't have it. It's because traits change and die off through inheritance. We even see this with Naruto. Naruto didn't inherit Kushina's unique chakra despite the fact that she was his mother. Nor did any of Hashirama's descendants inherit his power or his Mokuton. Simply being a descendant of the Sage doesn't mean your going to be a BAMF.
Also, that doesn't mean everyone is either 'Senju' or 'Uchiha'. The Sage and his sons were both members of the Otsutsuki clan. Any sons and grandsons would have carried on the Otsutsuki name, while any daughters would have married into other clans (Likely non-shinobi families, unless they committed incest). Eventually, the Otsutsuki clan went extinct, which means that somewhere down the line male descendants of both the Sage's sons either died before they could have offspring, or only had daughters. Or they simply stopped using the name. Who knows? Either way, the Sage's blood got around into many different families, not just the Senju and the Uchiha even if my theory turns out to be wrong. (There is the Uzumaki, for one.)