The ability to recognize and discriminate among others is a frequent assumption of models of the evolution of cooperative behavior. At the same time, cooperative behavior has been proposed as a selective agent favoring the evolution of individual recognition abilities. While theory predicts that recognition and cooperation may co-evolve, data linking recognition abilities and cooperative behavior with fitness or evidence of selection are elusive. Here, we provide evidence of a fitness link between individual recognition and cooperation in the paper wasp Polistes fuscatus. Nest founding females in northern populations frequently form cooperative multiple foundress nests and possess highly variable facial patterns that mediate individual recognition. We describe a dearth of cooperative nesting, low phenotypic diversity, and a lack of individual recognition in southern populations. In a common garden experiment, northern co-foundress associations successfully reared offspring while all cooperative southern groups failed to rear any offspring, suggesting a fitness link between individual recognition and successful cooperation. Consistent with a selective link between individual recognition and cooperation, we find that rates of cooperative co-nesting correlate with identity-signaling color pattern diversity across the species' range. Moreover, genomic evidence of recent positive selection on cognition loci likely to mediate individual recognition is substantially stronger in northern compared to southern P. fuscatus populations. Collectively, these data suggest that individual recognition and cooperative nesting behavior have co-evolved in P. fuscatus because recognition helps mediate conflict among co-nesting foundresses. This work provides evidence of a specific cognitive phenotype under selection because of social interactions, supporting the idea that social behavior can be a key driver of cognitive evolution.