The branches have to come from somewhere; Otherwise there wouldn’t be an Uchiha, Hyuga, or Senju clan at all since they’re all Otsutsuki. At some point, a group of people within that clan decided to branch off and start their own identity.kaguya Ōtsutsuki Hagoromo Ōtsutsuki and Hamura Ōtsutsuki
lol Ōtsutsuki is the clan if there was one, your misunderstanding.
But if Haruno or Hatake origins are expanded upon in Boruto (which I hope they aren’t) then OP could be right.