Detecting dominance relationships, within and across species, provides a clear fitness advantage because this ability helps individuals assess their potential risk of injury before engaging in a competition. Previous research has demonstrated that 10- to 13-mo-old infants can represent the dominance relationship between two agents in terms of their physical size (larger agent = more dominant), whereas younger infants fail to do so. It is unclear whether infants younger than 10 mo fail to represent dominance relationships in general, or whether they lack sensitivity to physical size as a cue to dominance. Two studies explored whether infants, like many species across the animal kingdom, use numerical group size to assess dominance relationships and whether this capacity emerges before their sensitivity to physical size. A third study ruled out an alternative explanation for our findings. Across these studies, we report that infants 6–12 mo of age use numerical group size to infer dominance relationships. Specifically, preverbal infants expect an agent from a numerically larger group to win in a right-of-way competition against an agent from a numerically smaller group. In addition, this is, to our knowledge, the first study to demonstrate that infants 6–9 mo of age are capable of understanding social dominance relations. These results demonstrate that infants’ understanding of social dominance relations may be based on evolutionarily relevant cues and reveal infants’ early sensitivity to an important adaptive function of social groups.