Well the idea is that while Mifune trained harder and became stronger, Hanzo slowly degraded himself into a weaker shinobi. This was stated more than once in their fight I believe. However, I don't personally feel that Mifune gets the credit he earned in that fight, or at least not enough credit. From what we know he: blocked a sly trick that Hanzo complimented on (showed kenjutsu abilities), stopped Hanzo from forming any hand seals and whatever jutsu he was about to use (his primary tactic), and both cut through the Salamander's mouth and sliced Hanzo's weapon into 2 (showing the power of his sword and further kenjutsu skills).
Conclusion: Hanzo was an immensely fast and powerful samurai that relied on his sharp sword as his over all power.
Now I feel that most people don't see him in a stronger light because he held the basic "speed and CQC" stereotype. But in truth, being able to stop your opponent from using hand seals can cut their amount of jutsu by a lot depending on what their move set is. What if Hanzo was about to use a trump card, but couldn't specifically because Mifune stopped him? IMO that's his true power along with his already renown sword skills.