This would be a very close fight and much closer than Kakashi VS Danzo. Jiraiya has the advantage when you look at it from an elemental perspective. Fire beats wind so Danzo's wind jutsus aren't going to be as effective as they could be. Danzo's cursed seal will also be ineffective since Jiraiya is not a CQC brawler. Baku, Danzo's trump card, while very powerful, can be easily fodderized by a katon or Yomi Nuri to sink it.
With most of Danzo's moveset countered by Jiraiya easily, I struggle to think of a way with which Danzo can win. Danzo's saving grace is sharingan precognition and 11 mintues of true immortality but Jiraiya can keep on making Danzo waste izanagi by hitting him with various jutsus along with all his toads. In other words, Danzo will get beat up pretty badly and end up wasting all of his eyes just to survive. Danzo could try to close the distance between them so that they will be more even but I think Jiraiya is faster especially with sage mode. Also like you said, Jiraiya can use his hair to defend himself from wind bullets, weapons and if Danzo gets close, he will be pushed back by Jiraiya's hair.
Still Danzo can hold his own against Jiraiya and may push him to the edge. If Jiraiya decides to use the frog song, it is easily countered by using wind jutsu to distort the sound waves. Danzo and Temari are the very few ninjas in the Narutoverse that have a legit counter to sound based genjutsu. Izanagi should work in dispelling genjutsu too but Danzo need not waste it on the frog song. Also Danzo can easily hold his own if not disable some of the frog summons with his powerful wind attacks. Still that would prove to be hard if Jiraiya uses all of his toads at once if he also decides to use his own set of attacks along with Ma and Pa's.
Once Danzo has used up all his izanagi, he could force Jiraiya to draw with him by using his reverse tetragram seal but I think Jiraiya can reverse summon himself in time in order to avoid it.
I say Jiraiya wins with mid-high difficulty. After reading Naruto VS Gaara where I saw a small fraction of Jiraiya's power, I now acknowledge him to have some of the best fire power in the manga.