Purpose Telemedicine use during the COVID-19 pandemic among financially distressed patients with cancer, with respect to the determinants of adoption and patterns of utilization, has yet to be delineated. We sought to systematically characterize telemedicine utilization in financially distressed patients with cancer during the COVID-19 pandemic. Methods We conducted a cross-sectional analysis of nationwide survey data assessing telemedicine use in patients with cancer during the COVID-19 pandemic collected by Patient Advocate Foundation (PAF) in December 2020. Patients were characterized as financially distressed by self-reporting limited financial resources to manage out-of-pocket costs, psychological distress, and/or adaptive coping behaviors. Primary study outcome was telemedicine utilization during the pandemic. Secondary outcomes were telemedicine utilization volume and modality preferences. Multivariable and Poisson regression analyses were used to identify factors associated with telemedicine use. Results A convenience sample of 627 patients with cancer responded to the PAF survey. Telemedicine adoption during the pandemic was reported by 67% of patients, with most (63%) preferring video visits. Younger age (19-35 age compared to ≥ 75 age) (OR, 6.07; 95% CI, 1.47-25.1) and more comorbidities (≥ 3 comorbidities compared to cancer only) (OR, 1.79; 95% CI, 1.13-2.65) were factors associated with telemedicine adoption. Younger age (19-35 years) (incidence rate ratios [IRR], 1.78; 95% CI, 24-115%) and higher comorbidities (≥ 3) (IRR; 1.36; 95% CI, 20-55%) were factors associated with higher utilization volume. As area deprivation index increased by 10 units, the number of visits decreased by 3% (IRR 1.03, 95% CI, 1.03-1.05). Conclusions The rapid adoption of telemedicine may exacerbate existing inequities, particularly among vulnerable financially distressed patients with cancer. Policy-level interventions are needed for the equitable and efficient provision of this service.