Severe acute respiratory syndrome coronavirus type 2 (SARS-CoV-2) virus is the cause of coronavirus disease 2019 (COVID-19). It has caused millions of infections and deaths globally over a 2-year period. Some populations including those living with HIV and/or cancer are reported to be at a higher risk of infection and severe disease. HIV infection leads to a depletion of CD4+ T cells which impairs cell-mediated immunity and increases the risk of malignancies such as Kaposi sarcoma (KS) and viral infections such as SARS-CoV-2. However, several other factors including level of immunosuppression and chemotherapy may also affect the immune response against SARS-CoV-2. In this study, we investigated factors affecting SARS-CoV-2-specific T cell immunity towards the spike, nucleoprotein, membrane protein, and other open reading frame proteins in individuals with HIV-associated KS. The KS patients were SARS-CoV-2 seropositive with detectable T cell responses, but had no history of symptomatic SARS-CoV-2 infection. We observed that the T cell responses increase from baseline levels during follow-up, with responses towards the NMO peptide pool being statistically significant. Low CD4 counts below 200 cells/μl were associated with lower SARS-CoV-2-specific T cell responses. Cancer chemotherapy and KS T staging did not have a significant effect on the T cell responses.