Of course he's stronger with it but are you saying that having Kamui hinders his actual character? I've never felt that way because of the way Kakashi himself uses Kamui, he's not one to spam unless it's necessary and look what he's been up against. Lately, he's been fighting 2 of the strongest shinobi to ever exist (Obito and Madara). At this point, Kamui is necessary especially he's used to outsmarting his opponent which is not easy against the strength and knowledge these two possess.
Kakashi is great with or without Kamui, I hardly ever take the "Kamui snipe" scenario in mind he's being put up against another character because it doesn't seem like something that Kakashi considers in a normal 1 v 1 battle scenario. He considered the idea with Kakuzu and Obito but that's about it I believe. But in most scenarios, Kakashi tries to get the job done without using MS.
Edit: It's the fans that take Kamui too far, not Kakashi himself.