Welfare recipients were often considered the least deserving of COVID-related support. Despite the recent attention paid to the impact of COVID-19 pandemic on mental health, few studies have explored the mental distress experienced by welfare recipients. This cross-sectional study on female Comprehensive Social Security Allowance recipients in Hong Kong aimed to explore their level of mental distress and its association with a range of risk factors specific to welfare recipients. Hence, 316 valid cases from a local community center responded to our online survey. We found that 52.3%, 23.4%, and 78% of the participants showed moderate to extremely severe depression, anxiety, and stress symptoms, respectively. A higher level of mental distress was associated with having a psychiatric diagnosis, poorer social, and greater concerns over disciplining children, the living environment, daily expenses and being infected by COVID-19. Unexpectedly, being married, having a permanent residence, and having a job were not significant protective factors for this group. The models explained 45.5%, 44.6%, and 52.5% of the overall variance in the level of depression, anxiety, and stress (p < 0.01), respectively. Our findings have important implications for supporting female welfare recipients during a public health crisis and may help frontline staff and professionals provide prompt assistance to this group in need.