The aim of this longitudinal cohort study, is to provide more insight into the pattern of brain abnormalities, and possible consequences for cognitive functioning, in patients with classic infantile Pompe disease. We included 19 classic infantile Pompe patients (median age last assessment 8.9 years, range 1.5–22.5 years; 5/19 CRIM negative), treated with ERT. Using MR imaging of the brain (T1, T2, and FLAIR acquisitions), we classified progression of brain abnormalities on a 12‐point rating scale at multiple time points throughout follow‐up. Additionally we noted specific white matter patterns and examined atrophy. Cognitive development was studied using Wechsler IQ assessments obtained by certified neuropsychologists. The association between age and cognitive functioning, and MRI ratings and cognitive functioning was assessed by linear regression models. All but one patient developed brain abnormalities. The abnormalities progressed in a similar pattern throughout the brain, with early involvement of periventricular white matter, later followed by subcortical white matter, gray matter structures, and juxtacortical U‐fibers. We found a significant decline (p < 0.01), with increasing age for full scale IQ, performance IQ and processing speed, but not for verbal IQ (p = 0.17). Each point increment in the 12‐point MRI rating scale was associated with a significant decline (3.1–6.0 points) in all the IQ index scores (p < 0.05). The majority of long‐term surviving patients in our cohort develop incremental brain MRI abnormalities and decline in cognitive functioning. This highlights the need for new therapies that can cross the blood–brain barrier in order to treat this CNS phenotype.