Background: Unpaid domestic work has been found to be negatively associated with mental health, especially among women, in previous studies but the measures of domestic work vary. The aim of this study was to elucidate the association between time spent in domestic work and mental health in the general population. Method: The study is based on 14,184 women and men aged 30–69 years who responded to a survey questionnaire in Central Sweden in 2017 (overall response rate: 43%). Multivariate logistic regression models, adjusting for age group, educational level, family status, employment status, economic difficulties, and social support, were used to study the association between hours spent in domestic work and depressive symptoms and self-reported diagnosed depression, respectively. Results: In total, 26.7% of the respondents reported depressive symptoms and 8.8% reported diagnosed depression. No independent associations between hours spent in domestic work and depressive symptoms were found. Among women, the lowest prevalence of depression was found among those who spend 11–30 h per week in domestic work. Among men, the prevalence of self-reported diagnosed depression was highest among those who spend 0–2 h per week in domestic work, but no other statistically significant associations between time spent in domestic work and depression were found. In addition, a strong dose–response relationship was found between experiencing domestic work as burdensome and both depressive symptoms and self-reported diagnosed depression among women and men. Conclusion: Investigating time spent in unpaid domestic work may not be sufficient to assess the association between exposure to domestic work and mental health. Conversely, strain in domestic work may be a more important factor contributing to the prevalence of poor mental health in the general population.