What do you mean by judged?
"People think that everyone from a clan is special"
Because they are. They have abilities that average shinobi don't. The ones you listed as examples are directly descended from the Otsutsuki.. that's special. The other ones not listed evolved over 1000 years of people heavily using, experimenting with, and progressing chakra/ninjutu. I'm pretty sure most of them are mutants by definition due to them developing specific chakra in order to preform clan specific jutsu/KKG... that is also special.
This is not to say that non-clan members aren't special. Sakura, Lee, Guy, Kakashi (never mentions a clan), Minato (also never specifically mentions a clan if I recall), etc are all characters that are assumed to be from either just a normal family, or lost clan that wasn't relevant enough to survive the times. They are all characters that either were or eventually became a cut above the normal standard of shinobi.
So yes. Judgement will be and has been made. Yes, clansmen are special. Specifically the ones directly descended from the Otsutsuki. That's just how it goes in this manga.