The extraordinary situation related to COVID-19 makes people worry about their health, family health, work, finances, and other daily activities. This condition can lead to social unrest, which has consequences for mental health problems. This study aims to determine the mental health consequences at the beginning of the COVID-19 pandemic in Indonesia. This is a cross-sectional study involving a target population aged 18 years and over who had access to electronic communication devices. An online questionnaire was randomly distributed and snowballed throughout 34 provinces in Indonesia. The study was conducted from 2 to 4 May 2020. Non-parametric and multivariate linear regression analyses were performed to identify factors associated with anxiety and depression. Two thousand seven hundred forty-three participants were involved in this study, with 69.16% female. In sum, 6.92% of participants had General Anxiety Disorder-7 (GAD-7) scores ≥ 10 for moderate-severe anxiety symptoms, and 8.57% had Patient Health Questionnaire-9 (PHQ-9) scores ≥ 10 for moderate-severe depressive symptoms. The multivariate linear regression analyses showed that the strongest factors influencing anxiety and depression were a history of mental illnesses, chronic illnesses, the group affected by layoffs or job seekers, unemployed, students, younger age group, living in a rented house, single, and female. In contrast, the lower and secondary education level seems to reduce the risk of depression compared to those with higher education levels. Anxiety and depression occur during the periods of activity restriction during the COVID-19 pandemic and are influenced by several modifiable and non-modifiable factors. There is an urgent need to emphasize vulnerable groups such as those with a history of illness, those affected by layoffs/looking for work, and the younger age group.