Early life adversity (ELA) increases the likelihood of later-life neuropsychiatric disorders and cognitive dysfunction. Importantly, ELA, neuropsychiatric disorders, and cognitive deficits all involve aberrant immune signaling. Microglia are the primary neuroimmune cells and regulate brain development. Microglia are particularly sensitive to early life insults, which can program their responses to future challenges. ELA in the form of maternal separation (MS) in rats alters later-life microglial morphology and the inflammatory profile of the prefrontal cortex, a region important for cognition. However, the role of microglial responses during MS in the development of later cognition is not known. Therefore, here we aimed to determine whether the presence of microglia during MS mediates long-term impacts on adult working memory. Clodronate liposomes were used to transiently deplete microglia from the brain, while empty liposomes were used as a control. We hypothesized that if microglia mediate the long-term impacts of ELA on working memory in adulthood, then depleting microglia during MS would prevent these deficits. Importantly, microglial function shifts throughout the neonatal period, so an exploratory investigation assessed whether depletion during the early versus late neonatal period had different effects on adult working memory. Surprisingly, empty liposome treatment during the early, but not late, postnatal period induced microglial activity changes that compounded with MS to impair working memory in females. In contrast, microglial depletion later in infancy impaired later life working memory in females, suggesting that microglial function during late infancy plays an important role in the development of cognitive function. Together, these findings suggest that microglia shift their sensitivity to early life insults across development. Our findings also highlight the potential for MS to impact some developmental processes only when compounded with additional neuroimmune challenges in a sex-dependent manner.