Background: The coronavirus disease 2019 (COVID-19) pandemic has resulted in a total upending of our daily lives. While anxiety and depression were frequently reported among the general population, the pandemic’s impact on patients with mental health problems remains unknown. Methods: A cross-sectional questionnaire survey involving 1,166 patients was conducted at one psychiatric hospital and one mental health clinic. Results: Symptom deterioration was reported in 23% to 34% of the patients and 9% to 20% reported increase in drug dosage. No significant differences were reported in these items among diagnostic categories. Patients with F3 (mood disorders) reported more psychological stress during the pandemic’s beginning and during the emergency. Patients with F2 (schizophrenia, schizotypal, and delusional disorders) did online shopping and meetings less frequently, and reported poorer adherence of 3C’s, while mask management was stricter in patients with F4 (neurotic, stress-related, and somatoform disorders). Symptom deterioration was significantly associated with increase in drug dosage, new physical symptoms, anxiety unrelated to COVID-19, stress at the beginning of pandemic, stress during the ‘state of emergency’, poor adaptability to environmental change, daily life changes, decrease in sleeping time, and decrease in time spent outside. Conclusion: One third of patients reported symptom deterioration during the pandemic, which was associated with stress and daily life changes. Patients with good adaptability to environmental changes might resilient against symptom deterioration. Providing continuous support to help patients manage their daily life in this COVID-19 era may minimize the risk of symptom deterioration.