There's no one cause, I would have to believe.
Obviously, the lack of compassion in a person's life (abuse, large amount of negative emotions, or just the lack of care at all.) could easily contribute to a person not having compassion themselves. People tend to reflect the environments they come from, so if a person never really experienced compassion, how would they be able to accurately mimic it? (Because trait behaviors are often, I have learned, acquired through mimicking.)
However, some people don't hold compassionate even if they come from loving homes or a well off background. We could attribute that to some sort of innate selfishness or feeling of entitlement. (I.e. "I don't care about this hobo lying in the street. What has he done for me? We are not like each other." That's a poor example, I know, but you should get the picture. >.>)
And some people just don't care. It's like the selfishness factor I spoke of just before, but it's also different. It's a factor that's not born from any particular sense of self importance, but rather just completely and utterly denying the fact that other people matter at all. Again, this stems from no emotion or negative mindset, but rather just a great apathy. (I.e., take the last example, and replace "what has he done for me", with "he does not affect me."
That's what I have seen as causes, at least. Hope this helped.