Human beings are highly familiar over-learnt social targets, with similar physical facial morphology between perceiver and target. But does experience with or similarity to a social target determine whether we can accurately infer emotions from their facial displays? Here, we test this question across two studies by having human participants infer emotions from facial displays of: dogs, a highly experienced social target but with relatively dissimilar facial morphology; panins (chimpanzees/bonobos), inexperienced social targets, but close genetic relatives with a more similar facial morphology; and humans. We find that people are more accurate inferring emotions from facial displays of dogs compared to panins, though they are most accurate for human faces. However, we also find an effect of emotion, such that people vary in their ability to infer different emotional states from different species’ facial displays, with anger more accurately inferred than happiness across species, perhaps hinting at an evolutionary bias towards detecting threat. These results not only compare emotion inferences from human and animal faces but provide initial evidence that experience with a non-human animal affects inferring emotion from facial displays.