There were over 10,000 marines on Enies Lobby alone and millions of pirates on the seas. The majority of the world being civilians doesn't matter when there's more than enough people to fit the bill who are active. There's also zero chance that there are common civilians running around with the capacity for Conqueror's, as people who have that capacity are drawn towards doing exceptional things and being exceptional people.
Hundreds of OP characters and only ten of them are Haoshoku users so far. It's not this dramatic.
Because in the New World, simply possessing Haoshoku isn't enough to have the edge, you have to actually have stronger Haoshoku over others for it to be an edge. We simply haven't gotten to that point yet.
Have you even been paying attention? For a rookie like Luffy to display Haoshokue, that tells the world he is a serious contender, something not many expected from him. That doesn't mean he's a top dog, that means he has that undeniable potential, and that is why they gave him that attention. There will be endless foes in Luffy's way in the New World who have it, but the actual top dogs will be the ones with exceptional Haoshoku.