Background: The COVID-19 pandemic and the government’s emergency declarations in Japan may have influenced people’s mental health. In particular, among women, there are concerns about the occurrence of neuroses, such as depression and anxiety. Objective: This study aimed to identify the factors related to mental distress among women in Japan who were raising children amid the COVID-19 pandemic. Methods: An online survey was conducted in 2020 among 730 Japanese women raising preschoolers. The survey included questions about child-rearing, anxiety, and the 12-item General Health Questionnaire (GHQ-12). The average age of the respondents was 34.4 years (21–52 years), and 31.5% of the respondents were living in “Prefectures under Specific Cautions” areas. Results: The prevalence of clinically significant levels of mental distress according to GHQ-12 scores was 41.5%. The highest predictor of mental distress was “Maternal anxiety about child-rearing” (Odds Ratio = 3.27, 95% Confidence Interval: 2.41–4.45). Other factors that affected their mental condition included concerns about children’s health and avoiding visiting hospitals for treatment. Conclusion: Mothers raising preschoolers experienced increased mental distress because of pandemic-related closures and heightened maternal anxiety about child-rearing. Thus, women who are raising preschoolers and confined to their homes need to be provided mental healthcare.