Research has documented an overlap between people’s aversion toward nonsocial pattern deviancy (e.g., a row of triangles with 1 triangle out of line) and their social prejudice. It is unknown which processes underlie this association, however, and whether this link is causal. We propose that pattern deviancy aversion may contribute to prejudice by heightening people’s dislike of statistical minorities. Infrequent people in a population are pattern deviant in that they disrupt the statistical regularities of how people tend to look, think, and act in society, and this deviancy should incite others’ prejudice. Nine studies (N = 1,821) supported this mediation. In Studies 1.1 and 1.2, adults’ and young children’s nonsocial pattern deviancy aversion related to disliking novel statistical minorities, and this dislike predicted prejudice against Black people. Studies 1.3 and 1.4 observed this mediation when experimentally manipulating pattern deviancy aversion, although pattern deviancy aversion did not directly impact racial prejudice. Study-set 2 replicated the proposed mediation in terms of prejudice against other commonly stigmatized individuals (e.g., someone with a physical disability). Importantly, we also found pattern deviancy aversion to affect such prejudice. Study-set 3 provided additional support for the mediation model. Pattern deviancy aversion predicted prejudice dependent on group-size, for instance, greater racial prejudice in cases where Black people are the statistical minority, but decreased racial prejudice when Black people are the statistical majority. Taken together, these findings indicate that people’s aversion toward pattern deviancy motivates prejudice, and that this influence is partially driven by a dislike of statistical minorities.