People living with HIV (PLWH) could be at risk of blunted immune responses to COVID-19 vaccination. We investigated factors associated with neutralizing antibody (NAb) responses against SARS-CoV-2 and variants of concern (VOCs), following two-dose and third booster monovalent COVID-19 mRNA vaccination in Japanese PLWH. NAb titers were assessed in polyclonal IgG fractions by lentiviral-based pseudovirus assays. Overall, NAb titers against Wuhan, following two-dose vaccination, were assessed in 82 PLWH on treatment, whereby 17/82 (20.73%) were classified as low-NAb participants. Within the low-NAb participants, the third booster vaccination enhanced NAb titers against Wuhan and VOCs, albeit to a significantly lower magnitude than the rest. In the multivariate analysis, NAb titers against Wuhan after two-dose vaccination correlated with age and days since vaccination, but not with CD4+ count, CD4+/CD8+ ratio, and plasma high-sensitivity C-Reactive protein (hsCRP). Interestingly, an extended analysis within age subgroups revealed NAb titers to correlate positively with the CD4+ count and negatively with plasma hsCRP in younger, but not older, participants. In conclusion, a third booster vaccination substantially enhances NAb titers, but the benefit may be suboptimal in subpopulations of PLWH exhibiting low titers at baseline. Considering clinical and immune parameters could provide a nuanced understanding of factors associated with vaccine responses in PLWH.