Our aim was to explore the psychological impact of the initial 2019 Coronavirus (COVID-19) pandemic in a cohort of people living with HIV (PLWH), a population at increased risk of psychological distress. PLWH, treated at our unit, responded an online survey. Data on demographics, clinical and physical symptoms, contact history, as well as knowledge, concerns, precautionary measures and additional information required with respect to COVID-19 during the first phase of the pandemic in Italy were collected. Additionally, the Impact of Event Scale-Revised (IES-R) (identifying COVID-19 pandemic as specific traumatic life event) and the Depression, Anxiety and Stress Scale (DASS-21) were also administered. Out of 98 participants, 44 (45%) revealed from mild to severe psychological impact distress from COVID-19 according to IES-R. According to DASS-21, a lower percentage of significant levels of depression (14%), anxiety (11%) or stress (6%) were reported. Higher education, unemployment, number of perceived physical COVID-19 symptoms, concerns contracting COVID-19 and the situation regarding the pandemic in Italy, and the need of additional information to prevent COVID-19 infection were positively associated to psychological distress. Moreover, female gender, older age, recent HIV diagnosis and unawareness of own viremia were associated to a higher psychological distress. Almost half of our cohort experienced significant levels of distress related to the COVID-19 pandemic. Women, elderly patients and those with recent HIV diagnosis resulted to be the more psychological fragile subgroup. Our findings could help to identify patients in need of psychological interventions to improve wellbeing of PLWH.