Ok, Minato is actually the fastest on every single aspect, both space time jutsu and pure speed.
About the space time jutsu... well let's compare it to madara's, yes it is more limited, due to the fact that you can only teleport to the targeted place (where he has his seal on) while madara can go wherever he wants to. However Minato's jutsu is faster, he teleports there instantly, with no hand seals, no nothing, just puf and he's gone... while madara takes some time in order to teleport. And also, i don't remember which of them actually said that Minato won the speed battle between the 2 of them.
And for pure speed, Madara is out, clearly... we haven't seen him move that fast, just pure speed that is. As for Naruto and Minato... well Minato is faster (at least now, but i am sure Naruto will surpass him) it was stated by Bee or Yamato that he is not yet on the fourth's level... And remember how he was able to save naruto from madara's arms using only his speed.
And also remember, there were many shinobys that were compared to the fourth in terms of speed, but none of them were at his level, they might have been able to keep up to him at most, but there is no way anyone could surpass him... except Naruto, remember the whole new generation surpasses the previous.
Oh and remember that naruto is that fast because of his new form... Just imagine how fast he will be if he could combine his new SO6P mode with sage mode xd
And one more thing, Minato didn't have any special powerups, no tailed beast, no sharingan, and for as far as we know, no bloodline limit. So yea, he is the fastest lol xd