You are right to question it though. To my knowledge a body flicker technique would not be fast enough to avoid the speed of that attack which was right on top of him. Here are some questions I had about him as well.
Hanzou:
Tried to unite the five countries to bring about peace and fought a war to do so. During his war against the Leaf he takes on, defeats, and names the three legendary Sennin. Later he teams with Danzo from the Leaf to betray the Akatsuki. My first initial thought was why would you "buddy-up" and try to setup ambushes with a group you recently were in a war against? I realize that they were playing "quid pro quo" and if this were the only question I would dismiss it, but there are other inconsistencies, for example Hanzou's sudden inability to remember folks names. Hanzou could barely remember who Mifune was, despite specifically asking for his name when he spared his life.
You must be registered for see links
. In the same exchange with Mifune, Hanzo had a flashback to his own death. In the flashback, Hanzou struggled to remember Yahiko, another person he should have distinctly remembered. Nagato expressed his shock and dissapointment that this once formidable foe had declined
You must be registered for see links
. It doesn't seem reasonable that Hanzou, who went out of his way to know his most formidable adversaries names, would suddenly forget them. Another item that stretches belief was the body flicker technique he used to escape Gedo Mazo mid-attack
You must be registered for see links
. Now, think of this as a writer. Why was Hanzou's escape necessary? He did nothing noteworthy following the escape, but the escape itself was noteworthy because it speaks to the power that he had, a power that abandoned him shortly after confronting Nagato. Several main characters also questioned Hanzou's loss of power throughout the story. During the Pain arc, Jiraiya also expressed his disbelief upon hearing that Hanzou had been defeated. One could argue that this was a plot device to show how strong Pain was, but the fact that Pain also expressed surprise leads me to the conclusion that there is another factor at work. Changes also occurred with Hanzou's personality. In one instance, he seems completely unfazed at the appearance of a Rinnegan on the battlefield, and later he spends his time hiding in his compound and second guessing his decisions. People that are committed to bring about a war don't second guess themselves.