Actually he isn't that fast in gear4. He seems to be fast only because he is currently combining gear4 with gear2(we see smoke around him), but the gear4 mode is more for less rubber compression, rendering him like a sort of balloon, more flexible and more strength than in gear1 or 2(but less than in 3). However the bouncy effect leave him open like we've seen with Doflamingo in this chapter. Also, he probably can't even stretch anymore in this state judging by the rubber inflation
So no, all in all, gear4 has his weakness and it can't alone replace the other gears