I'm playing Fallout4 right now, go buy it, you'll have a blast. You don't need Fallout3, New Vegas, or the classic games to understand the plot or setting.
The engine doesn't have much influence over graphical capacity considering the engines can still provide upgraded meshes, aniostropic filtering, and depth of field. See, Skyrim didn't have two of those I listed. Skyrim didn't have Godrays. But hey, it's on the same engine so it must be the same!
Look at this beauty:
You must be registered for see images
Is there even references that I won't understand or is it completely good to dive in?
They'd explain to you or immerse you in Lore while you play. Yeah, if you've played the previous games, you'd understand some of the Lore immediately, but your appreciation of the game wouldn't falter from not experiencing it before.