While previous studies have reported olfactory dysfunction (OD) in relation to cardiovascular disease (CVD), few population-based studies have investigated whether such associations differ by sex. The purpose of this study was to identify the association between CVD and its risk factors with OD, and the sex-associated differences within the general population. We examined 20,016 adults aged 40 and older from the Korean National Health and Nutrition Examination Survey. All subjects reported on their history of OD. CVD and its risk factors included coronary artery disease (CAD), stroke, hypertension, diabetes, obesity, abdominal obesity, and hypertriglyceridemia; logistic regression was used to analyse their associations with OD, and additive interaction was used to analyse the interaction between risk factors and sex. In males, CAD was more likely to be associated with OD (odds ratio [OR] 1.81, 95% confidence interval [CI] 1.05–3.14), whereas abdominal obesity was associated with OD in females (OR 1.39, 95% CI 1.06–1.84).Additive interaction were observed between abdominal obesity and female sex with a relative excess risk of interaction of 0.45 (95% CI 0.26–0.63). Our findings suggest the relationship between OD and CVD and its risk factors and sex-associated differences among middle-aged and older adults.