The BM serves as a blood‐forming organ, but also supports the maintenance and immune surveillance function of many T cells. Yet, in contrast to other organs, little is known about the molecular mechanisms that drive T‐cell migration to and localization inside the BM. As BM accumulates many CXCR3‐expressing memory CD8+ T cells, we tested the involvement of this chemokine receptor, but found that CXCR3 is not required for BM entry. In contrast, we could demonstrate that CXCR4, which is highly expressed on both naive and memory CD8+ T cells in BM, is critically important for homing of all CD8+ T‐cell subsets to the BM in mice. Upon entry into the BM parenchyma, both naïve and memory CD8+ T cells locate close to sinusoidal vessels. Intravital imaging experiments revealed that CD8 T cells are surprisingly immobile and we found that they interact with ICAM‐1+VCAM‐1+BP‐1+ perivascular stromal cells. These cells are the major source of CXCL12, but also express key survival factors and maintenance cytokines IL‐7 and IL‐15. We therefore conclude that CXCR4 is not only crucial for entry of CD8+ T cells into the BM, but also controls their subsequent localization toward BM niches that support their survival.