From Scientific American:
Child molestation and rape top the social taboo list, according to a survey of 186 people between the ages of 18 and 47, and smoking marijuana ranks lowest among the 19 choices of forbidden behavior. In the middle—worse than robbing a bank but better than spousal murder—lies incest between brothers and sisters. Given the deleterious genetic impacts of offspring from such mating, some researchers have suggested that there may be an evolved mechanism designed to prevent that from occurring. And now evolutionary psychologist Debra Lieberman of the University of Hawaii–Honolulu believes she may have elicited some of its functions from this simple questionnaire.
Many animals show such “kin radar.” By mixing siblings in a litter, for example, scientists have shown that animals that grow up together appear to avoid mating, whether genetically related or not, largely based on recognizing specific smells. The evolutionary psychologists hypothesize that some form of mental mechanism assesses various cues to come up with an estimate of how related two people are. “The real question is: What are these cues?”