Background: This study sought to evaluate COVID-19 associated physical and mental health symptoms among adults with allergies compared to the general U.S. adult population. Methods: Data for these analyses were obtained from the publicly available COVID-19 Household Impact Survey, which provides national and regional statistics about physical health, mental health, economic security, and social dynamics among U.S. adults (ages 18 and older). Data from 20–26 April 2020; 4–10 May 2020; and 30 May–8 June 2020 were included. Our primary outcomes for this analysis were physical and mental health symptoms experienced in the last seven days. The primary predictor was participants’ self-report of a physician diagnosis of an allergy. Results/Discussion: This study included 10,760 participants, of whom 44% self-reported having allergies. Adults with allergies were more likely to report physical symptoms compared to adults without allergies including fever (aOR 1.7, 95% CI 1.44–1.99), cough (aOR 1.9, 95% CI 1.60–2.26), shortness of breath (aOR 2.04, 95% CI 1.71–2.43), and loss of taste or sense of smell (aOR 1.9, 95% CI 1.58–2.28). Adults with allergies were more likely to report feeling nervous (cOR 1.34, 95% CI 1.13, 1.60), depressed (cOR 1.32, 95% CI 1.11–1.57), lonely (cOR 1.23, 95% CI 1.04–1.47), hopeless (cOR 1.44, 95% CI 1.21–1.72), or having physical reactions when thinking about COVID-19 pandemic (cOR 2.01, 95% CI 1.44–2.82), compared to those without allergies. During the COVID-19 pandemic, adults with allergies are more likely to report physical and mental health symptoms compared to individuals without allergies. These findings have important implications for diagnostic and treatment challenges for allergy physicians.