You must be registered for see images
From this picture, it seems whatever his relationship with Kaidou is, he's pissed at him. Somewhere along the line, Kaidou or his subordinates did something in Wano that warranted Kinemon, Momonosuke and one other person, Kanjorou. Their main objective was something situated in Zou (the next island, where Law's crew is). Somewhere along the line they got shipwrecked, and end up on Dressrosa, where Momonosuke was kidnapped, and Kanjurou sacrificed himself, letting himself get captured, so that Kinemon could catch a ship to Punk Hazard and retrieve Momo.
So unless there's something related to the Underground and Smile Production in Zou, there's no reason to assume the Samurai's objective had anything to do with the Smile Factory in Dressrosa, Punk Hazard or even DoFlamingo.
Since both the Samurai and Law want to take out Kaido first, I'm assuming there's a direct connection between Zou and Kaidou, maybe it's even his base island. I find it strange that Kinemon would go on a mission outside of Wano with his small child, and Kanjurou's sacrifice might denote something more than just friendship. And again, Kinemon seemed pissed, but Momo just seemed shocked. In his mind, I think he just recognized the name as the target of his mission, whereas Kinemon knows something more. I believe the Samurai know something about Momonosuke, and, for some reason, they felt it was better to have him leave the country rather than stay there more protected.
I too believe him eating the dragon fruit was just a coincidence, and if he really is worth, it is not because of that.
Kinemon's story:
You must be registered for see links