His upset because Roger was a man who also loved freedom and did whatever he wanted and angered the World Government. One of the main aims of the Revolutionary army is to anger & destroy the world government.
We can also see his remorse, when he gets the news of Whitebeard passing away. His disturbed clearly and states:
" The sea's are gonna be rough now with only 3 Yonkou, its time to gather up all our commanders Iva"