I'm not really sure there is a 'genetic' factor to it. You have to be careful of 'genetic' explanations, as they are often a sort of snake oil explanation. Since we know genetics plays a huge role in our biochemistry and everything else - we like to make the assumption that phenomena with unknown origins/explanations are genetic.
What we do know is that if two people are raised together, they are highly unlikely to partner later on in life. Childhood friends, in reality, rarely end up getting married for much this reason. It is suspected that this is a sort of natural way to try and drive people to seek those outside of the close family (and increase the likelihood of introducing different genes) - but it isn't known for certain.
What is also known is that the occasional 'inbred' relationship is not necessarily destructive. While it is over multiple generations - if the parents were from distinct enough groups, then any kids are highly unlikely to have problems (maternal diet has a greater influence).
The campaign against family relationships was largely launched by the Catholic Church to break up the power of royalty and aristocracy. It was customary for cousins of powerful families to be married to each other to produce heirs. Marriages with other powerful families were done not just for political brokering of power but also to keep 'new blood' running in the family. Without any knowledge of DNA, people knew that families who didn't allow outsiders into the reproductive chain got a little ... different ...
The Church, seeking to solidify its own power, largely broke up the royalty structures by teaching that it was a sin to marry someone who was closer than something like three vectors.
Not that I advocate marrying into your family - but it's not nearly as massive of an issue as it has been made out to be.