Few studies have addressed the impact of maternal mild/asymptomatic SARS-CoV-2 infection on the developing neonatal immune system. In this study, we analyzed umbilical cord blood and placental chorionic villi from newborns of unvaccinated mothers with mild/asymptomatic SARS-CoV-2 infection during pregnancy using flow cytometry, single-cell transcriptomics, and functional assays. Despite the lack of vertical transmission, levels of inflammatory mediators were altered in cord blood. Maternal infection was also associated with increased memory T, B cells, and non-classical monocytes as well as increased activation. However, ex vivo responses to stimulation were attenuated. Finally, within the placental villi, we report an expansion of fetal Hofbauer cells and infiltrating maternal macrophages and rewiring towards a heightened inflammatory state. In contrast to cord blood monocytes, placental myeloid cells were primed for heightened antiviral responses. Taken together, this study highlights dysregulated fetal immune cell responses in response to mild maternal SARS-CoV-2 infection during pregnancy.