According to Narutopedia, Hashirama died in some war (not one of the 4 big ones though), sometime after Tsunade was born. He's actually the one who taught her how to gamble. So I guess the First Hokage must've been just as much of a sucker.
It was during one of the ninja wars. I don't think it was in battle, because seriously, the guy has the power of wood.
For his brother to have become Hokage, he must have died quite young. Hashirama did not even get around to make all of the administrative set ups of the village. Maybe he was severely injured during the fight at the Valley of the End and died later... Maybe he was assassinated?