We identified associations between self-reported olfactory dysfunction (OD) and dietary attributes in participants aged ≥40 years (n = 6,356) from the nationally representative 2011–2014 National Health and Nutrition Examination Survey (NHANES). The chemosensory questionnaire and 24-h dietary recalls were administered by trained interviewers. OD was defined as self-report of either smell problems in the last year, worse smell relative to age 25, or perceiving phantom odors. Dietary outcomes included Healthy Eating Index 2015 score (HEI) with adequacy and moderation components (higher scores indicated higher diet quality), dietary diversity, energy density, and intake of major food groups. Survey-weighted linear regression models estimated OD–diet associations, adjusting for socio-demographic, lifestyle, and clinical factors. Adjusted mean difference (95% CI) between those with versus without OD, showed that adults with OD had significantly lower HEI moderation score (−0.67 (−1.22, −0.11)) and diets higher in energy density (0.06 (0.00, 0.11)), and percent energy from saturated fat (0.47 (0.12, 0.81)), total fat (0.96 (0.22, 1.70)), and added sugar (1.00 (0.33, 1.66)). Age and sex-stratified analyses showed that younger females (40–64 years) primarily accounted for the associations with diet quality and total/saturated fat intake. These findings inform dietary screening and recommendations for adults who report OD, including those experiencing transient or persistent smell loss with COVID-19.