Let's recap for a second, shall we?
Before they even entered Grandline:
1. They missed the route and ended up on Calm Belt, where they were "attacked" by Sea Kings thousands of times bigger than their ship. (A caravel...)
2. They almost died crashing against the sides of the Red Line, when entering the Reverse Mountain
3. There was a huge ass Whale blocking their path out of Reverse Mountain.
At Grandline:
4. They passed through an area so twisted in weather conditions that snow was mixed with lightning and rain and huge waves.
5. Their first island's habitants were a Bounty Hunter organization. If you check the mountains behind Whiskey Peak, they're not cactus, they're graveyards of pirates.
6. Their second island was stuck in time. It still had freakin' Dinosaurs. To top it all, 2 GIANTS were wrestling there. And the cherry on top, even more powerful members of that organization followed them there.
7. They encountered a freakin' gargantuan cyclone. O_O Not only that, it is said to be unpredictable. They were saved by dead-bed Nami, because she is just awesome.
8. Their third island was a Winter Island inhabited by giant Polar Bear-like rabbits with serious bad attitude. Not to mention the humans living on that island hated pirates at that point, and there was only 1 doctor (Which was said to be a crazy witch). Btw, remember why they needed a doctor in the first place? Nami had contracted a disease that was actually extinct already. (From their island number 2)
9. They shifted their route so there is no way telling if their other islands would be equally danger, but Alabasta, although they had to fight an entire organization and a Shichibunkai, that was because there was a revolution. Even though, the timing counts, and even peaceful islands could become war zones in instances, in the Grand Line, and that is what happened.
10. A huge galleon fell from the sky. Nuff' said.
11. They ended up in a all-pirate island, Jaya. You know how dangerous that is? If you're not a badass, you will be killed. Most pirates aren't badass, and most were killed.
After that they traveled the Knock Up Stream. What is the Knock Up Stream? It's a gigantic current that shoots upwards into the sky. And we found that they had only a slight chance of being successful.
12. Oh my, what is this? A freakin sky island 10.000 meters in the air. No biggie right? They were so weak even a lone skylander almost wiped them out. Again, a once peaceful island became a warzone. No biggie again, right?
13. Previously they had already encountered a Pirate Crew (The gorilla one). Now they encountered Foxy and had to do a Davy Back Fight. They could have ended right there.
14. Water 7 would have been peaceful if it weren't for robbers who knocked Usopp out cold and a conspiracy with Robin. So they had to travel to Ennies Lobbies. Other Pirate crews would have a more peaceful ride, maybe.
15. Thriller Bark and the Florian Triangle. Nuff' said.
16. Shabaody Archipelago. Really? Between the criminals on every corner and the abductions, it's a reunion spot for every major Pirate. How dangerous can it be. Not to mention that to travel to the next part of Grandline, one has to travel underwater, which is a path only 30% of the people survive.
17. There are many more dangers in the Grand Line. To name a few, Boin Archipelago, a Man-eating ISLAND, Kuraigana Island, filled with Humandrills that were exposed to vicious war, etc etc.
So, not dangerous? Don't make me laugh.