In the face of the COVID-19 pandemic and the exceptional situation that has been experienced on a global scale since 2020, it is essential to assess the impact of COVID-19 in several areas and domains. Therefore, this research seeks to evaluate the impact of COVID-19 on work-related quality of life (WRQoL) in a Portuguese-speaking sample, through the lens of sexual orientation. One thousand, five hundred and seventy-seven individuals participated in this study, of which 1396 (88.5%) self-identified as heterosexual, 95 (6.0%) as gay or lesbian, and 87 (5.5%) as bisexual. Participants responded to the “Work-Related Quality of Life” scale, the “Fear of COVID-19” scale, and the “Negative Impact of COVID-19” scale. Bisexuals scored higher on “Fear of COVID-19” and “Negative Impact of COVID-19” than heterosexual, and gay, or lesbian participants. Differences between sexual orientations for all dimensions of WRQoL were found: heterosexual participants scored higher on general well-being, home–work interface, career satisfaction, working conditions, and lower on stress at work, compared to bisexual, and gay, or lesbian participants. Gay or lesbian participants scored lower than heterosexual and bisexual participants on career satisfaction and working conditions. Sexual orientation, the fear of COVID-19, and the negative impact of COVID-19 were significant predictors of overall WRQoL (explaining 13% of variance). Moderation analysis also showed that sexual orientation is a significant moderator of the association between the fear of COVID-19, the negative impact of COVID-19, and WRQoL. LGB people (especially bisexuals) suffer more severe impacts of COVID-19 and have lower WRQoL than heterosexual people. Inevitably, this has consequences in terms of mental health and overall quality of life for sexual minorities, thus reinforcing the need to adopt inclusive policies in organizations and companies to improve their WRQoL.