Minato and Tobi ran at each other. Minato on the other hand, had to make Tobi go on the offensive to make sure he was tangible, or atleast took that gamble. If he would've attacked Tobi head on, Tobi might've still been intangible so his effort would've failed.
Instead, he threw his kunai at Tobi which Tobi managed to phase through, then as Tobi was going to grab him, he FTGed to the Kunai that had just passed through Tobi and hit him with a Rasengan from behind or above as it seemed. Out of Tobi's line of sight. It was unexpected. Minato planned out that gamble as a counter attack that worked.
Tobi's reflexes though are extremely good though. Perhaps a tad below Minato's or up there with his. FTG just happened to be instantanious teleportation that is one of Tobi's S/T weaknesses.
As for him warping Sasuke at the Kage summit, it could've been plot. Though it's also possible that this time it was shown at its actual speed. Other times, it shows the entire absorbing process. It could be like those times when the manga allows us to see what's happening when in reality, they're going at extreme speeds that others at that spot wouldn't be able to see. His S/T is slower than FTG but could still be fast enough to not be noticed by other people. Just like some don't really get the chance to react when being warped. Minato's FTG was instant. Faster than Tobi's S/T. But he still had the reflexes to actually FTG to avoid being warped. Some people might be able to notice his S/T and react.
Other people like Kankuro at the summit noticed it was Tobi's S/T when he warped Karin to heal Sasuke. This proves that it's not instant but it is still fast. It could be seen by others but the process would be a few seconds. It still requires one to have good reflexes to avoid being warped.
Though I do believe that the battle took seconds or a few minutes but was shown in a motion for us to be able to see as the readers.