Manga disagrees. Both the Warlords and Yonko are chosen by the World Government in order to maintain the balance between the three. The four Yonko, without their crew, by themselves, are equal to the 7 Warlords, who are equal to the Marine force. Which isnt even that hard to believe, rookies Luffy, Law and Kidd who were below the Shichibukai minus Crocodile were trashing Marine platoons with little difficulty, it's not really THAT difficult to fathom that the Warlords and Yonko could do the same. The Marine force rises up into the thousands...thousands of fodder. The only notables are the Admirals, Fleet Admirals and some of the VAs really.