There is enormous variation in the extent to which fetal Zika virus (fZIKV) infection affects the developing brain. Despite the neural consequences of fZIKV infection observed in people and animal models, many open questions about the relationship between infection dynamics and fetal and infant development remain. To further understand how ZIKV affects the developing nervous system and the behavioral consequences of prenatal infection, we adopted a nonhuman primate model of fZIKV infection in which we inoculated pregnant rhesus macaques and their fetuses with ZIKV in the early second trimester of fetal development. We then tracked their health across gestation and characterized infant development across the first month of life. ZIKV-infected pregnant mothers had long periods of viremia and mild changes to their hematological profiles. ZIKV RNA concentrations, an indicator of infection magnitude, were higher in mothers whose fetuses were male, and the magnitude of ZIKV RNA in the mothers’ plasma or amniotic fluid predicted infant outcomes. The magnitude of ZIKV RNA was negatively associated with infant growth across the first month of life, affecting males’ growth more than females’ growth, although for most metrics, both males and females evidenced slower growth rates as compared with control animals whose mothers were not ZIKV inoculated. Compared with control infants, fZIKV infants also spent more time with their mothers during the first month of life, a social behavior difference that may have long-lasting consequences on psychosocial development during childhood.