There are several cellular and acellular structural barriers associated with the brain interfaces, which include the dura, the leptomeninges, the perivascular space and the choroid plexus epithelium. Each structure is enriched by distinct myeloid populations, which mainly originate from erythromyeloid precursors (EMP) in the embryonic yolk sac and seed the CNS during embryogenesis. However, depending on the precise microanatomical environment, resident myeloid cells differ in their marker profile, turnover and the extent to which they can be replenished by blood-derived cells. While some EMP-derived cells seed the parenchyma to become microglia, others engraft the meninges and become CNS-associated macrophages (CAMs), also referred to as border-associated macrophages (BAMs), e.g., leptomeningeal macrophages (MnMΦ). Recent data revealed that MnMΦ migrate into perivascular spaces postnatally where they differentiate into perivascular macrophages (PvMΦ). Under homeostatic conditions in pathogen-free mice, there is virtually no contribution of bone marrow-derived cells to MnMΦ and PvMΦ, but rather to macrophages of the choroid plexus and dura. In neuropathological conditions in which the blood–brain barrier is compromised, however, an influx of bone marrow-derived cells into the CNS can occur, potentially contributing to the pool of CNS myeloid cells. Simultaneously, resident CAMs may also proliferate and undergo transcriptional and proteomic changes, thereby, contributing to the disease outcome. Thus, both resident and infiltrating myeloid cells together act within their microenvironmental niche, but both populations play crucial roles in the overall disease course. Here, we summarize the current understanding of the sources and fates of resident CAMs in health and disease, and the role of the microenvironment in influencing their maintenance and function.