What if pirates of his time are strong? How can it help you to prove that admirals of that time are stronger? I think you are overlooking the fact that Luffy's going to reach the same level as Roger. Do you think Oda would make the FA of Luffy's ara to be weaker than FA of Rogers ara? Admirals of Rogers time are same as current admirals until proven otherwise via feats or portrayal.
You know that the battle b/w Akainu and Aokiji was the first ever battle over the title of FA right? which proves that no one before Akainu was selected as FA based on strength. Before his battle with Aokiji,no one knows if Akainu was stronger than Aokiji or not. Yet higher ups suggested him to be FA which proves that what they concerns the most to select someone as FA is his leadership skills not just strength. A FA is an admiral with good leadership skills, even if he's stronger than his fellow admirals, He can never be on another level when compared to admirals.
Kong might be a special exception like Garp but we can't say that for sure as we know only a little about him. He's at least admiral level for all we know. Anything above admiral level would be nothing more than a mere speculation at this point.