Yeah, you should question this. Kakashi's only got a small pool of chakra, but then it became like he has an infinite source of chakra even before he got Naruto's cloak
I guess it could be plot but couldn't he also just be getting better at chakra control with it? Also sasuke was going blind with MS but maybe Susano'o and enton/amaterasu just strain the eye more than kamui. Just a thought