The COVID-19 pandemic has generated fear, panic, distress, anxiety, and depression among many people in Bangladesh.In this cross-sectional study, we examined factors associated with different levels of psychological impact as a result of COVID-19 in Bangladesh. From April 1 to 30, 2020, we used a self-administered online questionnaire to collect data from 10,609 respondents. Using the Impact of Event Scale-Revised to assess the psychological impact of the COVID-19 pandemic on respondents, we categorized the levels of impact as normal, mild, moderate, or severe. Ordinal logistic regression was used to examine the associated factors. The prevalence of mild, moderate, and severe psychological impact was 10.2%, 4.8%, and 45.5%, respectively. Multivariate analysis revealed that the odds of reporting normal vs mild, moderate, or severe psychological impact were 5.9 times higher for people living in the Chittagong Division, 1.7 times higher for women with lower education levels, 3.0 times higher among those who were divorced or separated, 1.8 times higher for those working full time, and 2.4 times higher for those living in shared apartments. The odds of reporting a psychological impact were also higher among people who did not enforce protective measures inside the home, those in self-quarantine, those who did not wear face masks, and those who did not comply with World Health Organization precautionary measures. Increased psychological health risks due to COVID-19 were significantly higher among people who experienced chills, headache, cough, breathing difficulties, dizziness, and sore throat before data collection.