Adherence to antiretroviral therapy (ART) is a complex and multi-determined process that is influenced by psychosocial variables. Although international studies have pointed to the adverse impact of HIV stigma, sexual stigma, and depression on ART adherence among men who have sex with men (MSM) with HIV, less is known about this association among Brazilians. We aimed to (a) evaluate indicators of depression, stigma related to HIV and homosexuality, and adherence to ART in a sample of Brazilian MSM living with HIV; (b) assess possible correlations between the variables analyzed, and (c) assess the impact of HIV and sexual stigma and depression on ART adherence. This cross-sectional study comprised 138 Brazilian MSM living with HIV as participants. Scales used included: a sociodemographic/clinical questionnaire, the questionnaire for assessment of adherence to antiretroviral therapy (CEAT-HIV), the Beck depression inventory (BDI-II), the internalized homophobia scale, and the HIV stigmatization scale. The mean adherence score was relatively high (78.83, within a range of 17–89 points). However, we observed inadequate ART adherence (CEAT-HIV < 75) in 28 (20.2%) respondents. Participants reported high scores for internalized sexual stigma, perceived sexual stigma in the community, and HIV stigma. Symptoms of depression were identified in 48.47% of participants. We found negative correlations between depression, HIV stigma, and treatment adherence, but not between sexual stigma and ART adherence. HIV-related stigma and sexual stigma were positively correlated with depression. Our regression analysis indicated that each year of age at diagnosis of HIV increased adherence by 0.22 points, on average. Each additional BDI-II score reduced adherence to ART by 0.20 points. The high prevalence of depression, HIV stigma, and sexual stigma, and their adverse effects on ART adherence and mental health, point to the need to implement evidence-based interventions to reduce sexual and serological stigma in the general population, as well as to mitigate the negative impacts of stigma on MSM living in HIV in Brazil. They also highlight the importance of periodically screening for these variables among MSM treated in Brazilian public health services, especially among those with inadequate adherence to ART.