Atrial fibrillation (AF) is associated with an elevated risk of cognitive impairment and dementia. The investigation of the cognitive sequelae and alterations of brain structure linked to AF is crucial to help address ensuing health care needs. In this study, we conducted a comprehensive neuropsychological and neuroimaging analysis of 1335 stroke-free individuals with AF (30% women, average age 69.1 years) and compared them with 2683 demographically and cardiovascular risk-matched controls (31% women, average age 69.1 years). Primary study outcomes were neuropsychological test scores and advanced magnetic resonance imaging (MRI) measures of gray matter morphology, gray and white matter microstructure, and white matter hyperintensity (WMH) load. Our analysis identified deficits in attention/executive function, information processing speed and reasoning in individuals with AF. These cognitive impairments were accompanied by a complex imaging profile suggestive of small vessel pathology: (1) reduced cortical thickness and gray matter volume in areas including primary motor, somatosensory and visual cortices as well as the orbitofrontal, lateral prefrontal, posterior insular, and temporal cortices; (2) increased extracellular free-water content in the anterior cingulate, insula, medial prefrontal cortex, medial temporal lobe, and precuneus; (3) widespread microstructural anomalies in the cerebral white matter, marked by lower fractional anisotropy (FA), higher mean diffusivity (MD) and extracellular free-water, and a higher burden of markers of small vessel disease (WMH load and peak width of skeletonized mean diffusivity). Crucially, brain structural differences statistically mediated the link between AF and cognitive performance. By integrating a multimodal analysis approach with extensive clinical and MRI data, our study highlights small vessel pathology as a possible unifying link between AF, cognitive decline and abnormal brain structure. These insights can inform diagnostic approaches and motivate the ongoing implementation of effective therapeutic strategies.