Oxytocin receptor (OXTR) and arginine vasopressin receptors (AVPR1a, AVPR1b, and AVPR2) are paralogous genes that emerged through duplication events; along the evolutionary timeline, owing to speciation, numerous orthologues emerged as well. In order to elucidate the evolutionary forces that shaped these four genes in placental mammals and to reveal specific aspects of their protein structures, 35 species were selected. Specifically, we investigated their molecular evolutionary history and intrinsic protein disorder content, and identified the presence of short linear interaction motifs. OXTR seems to be under evolutionary constraint in placental mammals, whereas AVPR1a, AVPR1b, and AVPR2 exhibit higher evolutionary rates, suggesting that they have been under relaxed or experienced positive selection. In addition, we describe here, for the first time, that the OXTR, AVPR1a, AVPR1b, and AVPR2 mammalian orthologues preserve their disorder content, while this condition varies among the paralogues. Finally, our results reveal the presence of short linear interaction motifs, indicating possible functional adaptations related to physiological and/or behavioral taxa-specific traits.

Hominin evolution is characterized by adaptive solutions often rooted in behavioral and cognitive changes. If balancing selection had an important and long-lasting impact on the evolution of these traits, it can be hypothesized that genes associated with them should carry an excess of shared polymorphisms (trans- SNPs) across recent Homo species. In this study, we investigate the role of balancing selection in human evolution using available exomes from modern (Homo sapiens) and archaic humans (H. neanderthalensis and Denisovan) for an excess of trans-SNP in two gene sets: one associated with the immune system (IMMS) and another one with behavioral system (BEHS). We identified a significant excess of trans-SNPs in IMMS (N=547), of which six of these located within genes previously associated with schizophrenia. No excess of trans-SNPs was found in BEHS, but five genes in this system harbor potential signals for balancing selection and are associated with psychiatric or neurodevelopmental disorders. Our approach evidenced recent Homo trans-SNPs that have been previously implicated in psychiatric diseases such as schizophrenia, suggesting that a genetic repertoire common to the immune and behavioral systems could have been maintained by balancing selection starting before the split between archaic and modern humans.

Domestication is of unquestionable importance to the technological revolution that has given rise to modern human societies. In this study, we analyzed the DNA and protein sequences of six genes of the oxytocin and arginine vasopressin systems (OXT-OXTR; AVP-AVPR1a, AVPR1b and AVPR2) in 40 placental mammals. These systems play an important role in the control of physiology and behavior. According to our analyses, neutrality does not explain the pattern of molecular evolution found in some of these genes. We observed specific sites under positive selection in AVPR1b (ω = 1.429, p = 0.001) and AVPR2 (ω= 1.49, p = 0.001), suggesting that they could be involved in behavior and physiological changes, including those related to the domestication process. Furthermore, AVPR1a, which plays a role in social behavior, is under relaxed selective constraint in domesticated species. These results provide new insights into the nature of the domestication process and its impact on the OXT-AVP system.

The general picture obtained is of similarity in cognitive genes between extinct and extant humans.