In some translations, it's what Roger said. In other translations, he said he was cornered on multiple occasions and i trust that one more because it's what Garp had always been known for from public view(he has been introduced as such back in Water7) and when top tier fights, the public always know, yet he wasn't introduced as someone capable to almost kill Roger in a 1 on 1 fight, but rater as someone who could corner him and as stated, you don't need to be as strong as someone to be able to corner him. Shiki also corned Roger pirates with his fleet and almost defeated him if not for weather conditions that saved Roger and we also know Roger is the reckless type who never turns his back just like Ace, so him finding himself in such situations isn't hard to believe
Not to mention, we also don't know the timeline, we don't know when it happened. For example, in his historic as a pirate, guys like Lucci or Crocodile would be known as guys who almost killed Luffy, on multiple occasions in Crocodile's case, Smoker will be known as someone who almost captured Luffy on multiple occasions, and so on and so on, yet they won't be even close to him as a pirate, well except Smoker maybe
On the other side, logic dictates that they weren't that close
1- WB who was his actual equal became the world strongest man infront of Garp after he died
2- Even after weakening much worse than Garp, he still retained the WSM title
3- Garp and Sengoku had to team up to defeat someone weaker than Roger in a long battle that destroyed half marineford
4- Shiki implied that Sengoku and Garp couldn't capture Roger