I think your underrating Tsunade's ability in taijutsu. She wouldn't have to struggle very hard in my mind. I mean, it's not looking good for Neji if he gets hit even once.
No, people are underrating Nejis ability in taijutsu. While he is not as old and has not had as much experience as Tsunade, He is a genius in combat and has dedicated his whole life to utilising the gentle fist taijutsu, which is the perfect counter for Tsunade's chakra/strength-based taijutsu, so she won't be able to use her super strength or healing techniques, while her chakra control will eventually be taken out.
Tsunade on the other hand is primarily a medic ninja and secondarily a combat ninja despite her combat prowess. She first had to learn, understand, and master medic-nin abilities and responsibilities before being able to use it to her advantage in combat. Her evasion skills are defintely top-class, but Neji's hand speed is second in the NV only to gai with his morning peacock, and not only that, but his footspeed happens to be greater than Tsunades by a whole point in the databooks.
Peoeple dont like it when it seems like a jounin level ninja could take out a kage, but this one is smiply due to match-up and restrictions. Give tsunade katsuya and she could pull off a win mid-diff.