The HIV-mediated decline in circulating CD4 T cells correlates with increased risk of active tuberculosis (TB). However, HIV/Mycobacterium tuberculosis (Mtb) co-infected individuals also have an increased incidence of TB prior to loss of CD4 T cells in blood, raising the possibility that HIV co-infection leads to disruption of CD4 T cell responses at the site of lung infection before they are observed systemically. Here we used a rhesus macaque model of SIV/Mtb co-infection to study the early effects of acute SIV infection on CD4 T cells in pulmonary Mtb granulomas. Two weeks after SIV co-infection CD4 T cells were dramatically depleted from granulomas, before significant bacterial outgrowth, disease reactivation as measured by PET-CT imaging, or CD4 T cell loss in blood, airways, and lymph nodes. Mtb-specific CD4 T cells, CCR5-expressing, in granulomas were preferentially depleted by SIV infection. Moreover, CD4 T cells were preferentially depleted from the granuloma core and lymphocyte cuff relative to B cell-rich regions, and live imaging of granuloma explants showed that SIV co-infection reduced T cell motility. Thus, Mtb-specific CD4 T cells in pulmonary granulomas may be decimated before many patients even experience the first symptoms of acute HIV infection.