The combination of COVID-19 outbreaks and nationwide lockdown led to an increased prevalence of psychological distress among the population, especially women, as they have to cope with greater family and work demands. We aimed to identify the factors contributing to psychological distress among Malaysian women during the COVID-19 pandemic. A cross-sectional study was conducted between October 2020 and April 2021, in a teaching hospital in Kuala Lumpur, Malaysia. A self-administered questionnaire was distributed among women, which consisted of (1) Participant’s demographics, (2) COVID-19 knowledge and awareness, (3) Depression, Anxiety, Stress Scale-21 (DASS-21), and (4) COVID-19-related anxiety. Chi-square test and univariate analysis were performed to determine the significant factors associated with psychological distress. The mean scores for knowledge, COVID-19 anxiety, and DASS-21 subcomponents were compared using the Mann–Whitney U test. A total of three hundred and thirty-eight women completed the survey. The majority of respondents demonstrated adequate knowledge (95.6%) on COVID-19. The proportion of our women who reported symptoms of depression, anxiety, and stress were 17.2%, 25.1%, and 0.9%, respectively, resulting in a prevalence of psychological distress of 27.8%. Low education level (p = 0.017), unemployment (p = 0.028), loss of income (p = 0.033), and hospital admission for surgical procedures (p = 0.021) were significantly associated with a higher psychological burden. A greater level of COVID-19 anxiety was found among Malays (p = 0.027), pregnant women (p = 0.013), and those who suffered a loss of income (p = 0.038) during this pandemic. The COVID-19 pandemic had a negative effect on women’s psychological wellbeing, especially those from the lower socio-economic background. Therefore, adequate information, as well as support, must be provided to the vulnerable groups during the ongoing pandemic, to lessen their psychological burden.