Atrial fibrillation (AF) is one of the risk factors for dementia. Female sex is an inconsistent risk factor for dementia after adjusting for age in the general population, and there lacks research on its impact in developing dementia in patients with AF. This paper aims to investigate whether female sex is a risk factor for dementia in AF patients. Data of patients with newly diagnosed AF between 2001–2013 were retrieved from Taiwan’s National Health Insurance Research Database. Exclusion criteria were: patients with incomplete demographic data, age < 20 years, rheumatic heart disease, hyperthyroidism, past valvular heart surgery, and a history of dementia. Propensity score matching (PSM) between sexes was performed, including comorbidities, medications and index date stratified by age. The primary outcome was a new diagnosis of dementia at follow-up. A total of 117,517 men and 156,705 women were eligible for analysis. After 1:1 PSM, both 100,065 men and women (aged 72.5 ± 12.5 years) were included for analysis. Dementia risk varied with age in women compared with men. The difference was negligible for ≤55 years (sub distribution HR (SHR) = 0.89, 95% CI 0.73–1.07), but increased between 56–65 years (SHR = 1.13, 95% CI 1.02–1.25), 66–75 years (SHR = 1.14, 95% CI 1.09–1.20), 75–85 years (SHR = 1.11, 95% CI 1.07–1.15) and >85 years (SHR 1.10, 95% CI 1.04–1.16) for females. This study establishes that female sex increases the risk of developing dementia compared to male sex in AF patients aged >56 years. However, the impact of female sex on dementia in AF patients differs between dementia types.