Why would people say that? I have not seen him display any racist tendencies in the manga. Is he prejudice? Yes. Is he racist? No.
People keep using the word racist, but they do not know what it means. Allow me to clarify:
The Uchiha clan is NOT a race. Therefore Tobirama's disdain towards them, is NOT racism.