Cognitive impairment (CI) occurs in 43 to 70% of multiple sclerosis (MS) patients at both early and later disease stages. Cognitive domains typically involved in MS include attention, information processing speed, memory, and executive control. The growing use of advanced magnetic resonance imaging (MRI) techniques is furthering our understanding on the altered structural connectivity (SC) and functional connectivity (FC) substrates of CI in MS. Regarding SC, different diffusion tensor imaging (DTI) measures (e.g., fractional anisotropy, diffusivities) along tractography-derived white matter (WM) tracts showed relevance toward CI. Novel diffusion MRI techniques, including diffusion kurtosis imaging, diffusion spectrum imaging, high angular resolution diffusion imaging, and neurite orientation dispersion and density imaging, showed more pathological specificity compared to the traditional DTI but require longer scan time and mathematical complexities for their interpretation. As for FC, task-based functional MRI (fMRI) has been traditionally used in MS to brain mapping the neural activity during various cognitive tasks. Analysis methods of resting fMRI (seed-based, independent component analysis, graph analysis) have been applied to uncover the functional substrates of CI in MS by revealing adaptive or maladaptive mechanisms of functional reorganization. The relevance for CI in MS of SC–FC relationships, reflecting common pathogenic mechanisms in WM and gray matter, has been recently explored by novel MRI analysis methods. This review summarizes recent advances on MRI techniques of SC and FC and their potential to provide a deeper understanding of the pathological substrates of CI in MS.