Introduction This study aims to examine (1) the impact of the pandemic phases on overall and preventable hospitalizations and emergency department (ED) visits, and (2) the effect of the pandemic on these outcomes within subgroup populations including gender, race, patients’ residence in health professional shortage areas (HPSA), and residence in a federal poverty level. Study Design We used electronic medical record (EMR) data for the year 2019 and 2020 from a large health system predominantly serving medically underserved patients in the South. We used a difference-in-differences approach to examine changes in weekly rates of overall and preventable hospitalizations and ED visits in the pandemic phase 1 (Mid-March to June of 2020) and phase 2 (July-September of 2020) compared to the same period in 2019 after adjusting for weekly outcome rates in the baseline period (January to Mid-March of 2020) compared to the same period in 2019. Results The study sample included 1.4 million hospitalizations and ED encounters. In phase 1 of the pandemic, there were significant reductions in overall (−108) and preventable (−75.3) hospitalizations, and overall (−408) and preventable (−306) ED visits when compared to the same period in 2019. In phase 2 of the pandemic, there were significant reductions in overall (−60) and preventable (−43) hospitalizations and in overall (−360) and preventable (−258) ED visits as compared to 2019. We found greater reductions in ED visits, both overall and preventable, during the early pandemic phases among Black patients than among White patients. Similar patterns in the reduction of ED visits were found in Black versus White patients within subgroups of women, men, and those residing in a HPSA and low-income areas. Discussion Substantial reductions in utilization were observed in Black patients in comparison to white patients and these differences persisted among men, women, and those living in underserved and low-income areas.