The coronavirus disease 2019 (COVID-19) pandemic has caused enormous psychological impact worldwide. We conducted a systematic review and meta-analysis on the psychological and mental impact of COVID-19 among healthcare workers, the general population, and patients with higher COVID-19 risk published between 1 Nov 2019 to 25 May 2020. We conducted literature research using Embase, PubMed, Google scholar and WHO COVID-19 databases. Among the initial search of 9207 studies, 62 studies with 162,639 participants from 17 countries were included in the review. The pooled prevalence of anxiety and depression was 33% (95% confidence interval: 28%-38%) and 28% (23%-32%), respectively. The prevalence of anxiety and depression was the highest among patients with pre-existing conditions and COVID-19 infection (56% [39%-73%] and 55% [48%-62%]), and it was similar between healthcare workers and the general public. Studies from China, Italy, Turkey, Spain and Iran reported higher-than-pooled prevalence among healthcare workers and the general public. Common risk factors included being women, being nurses, having lower socioeconomic status, having high risks of contracting COVID-19, and social isolation. Protective factors included having sufficient medical resources, up-to-date and accurate information, and taking precautionary measures. In conclusion, psychological interventions targeting high-risk populations with heavy psychological distress are in urgent need.