It was actually stated that it was his favorite island, not that it was his only island or that he was living on it.
Also, the Strawhats had no business in Fishman Island, Skypeia or even Dressrosa until they either got there because it was along the way or because someone took them there. Part of their adventure is simply enjoying themselves and simply adventuring, Zoro and Brook's interest in Wano is enough for them to go there. Momo and Kinemon is enough for them to go there.
They had business on FI because it was the only way to reach the NW
They had business on skypeia because their log pose led them to there, so they needed to go there
And Law took them to Dressrosa
But each and every one of those islands had a reason to go there, and like I said before there are at least 3 different destinations that have more priority than Wano. Whole cake island, Zou and Kaidou metal island.
If Wano is such a strong and independent samurai island that doesn't like outsiders and isn't familiar with outside culture very much, I doubt someone like Kaidou is chilling there because that wouldn't make a lot of sense.
Also how would the sh's pick their next island ? they are stuck with the 3 way log pose and unless they can find an eternal pose for Wano how would they navigate to there.
And why would Momo and Kin'nemon take them there ? Please come to our country that doesn't like outsiders and please guy with the green hair bring the katana that you stole from one of our people's heroes grave. There are more pressing matters on hand than a roadtrip to take momo and his caretakers home.