Poor mental health has been found to be more prevalent among those with cancer and is considered a public health crisis since COVID-19. This study assessed the impact of COVID-19 and cancer survivorship on mental health and investigated factors, including online patient-provider communications (OPPC; email/internet/tablet/smartphone), associated with poor mental health prior to and during the early COVID-19. Nationally representative Health Information National Trends Survey data during 2017–2020 (n = 15,871) was used. While the prevalence of poor mental health was high (40–42%), Difference-In-Difference analyses revealed that cancer survivorship and COVID-19 were not associated with poor mental health. However, individuals that used OPPC had 40% higher odds of poor mental health. Low socioeconomic status (low education/income), younger age (18–64 years), and female birth gender were also associated with poor mental health. Findings highlight the persistence of long-standing mental health inequities and identify that OPPC users might be those who need mental health support.