Background: The risk factors for impaired cognitive development after unilateral perinatal stroke are poorly understood. Non-verbal intelligence seems to be at particular risk, since language can shift to the right hemisphere and may thereby reduce the capacity of the right hemisphere for its originary functions. Pharmaco-refractory epilepsies, a frequent complication of perinatal strokes, often lead to impaired intelligence. Yet, the role of well-controlled epilepsies is less well-understood. Here, we investigated whether well-controlled epilepsies, motor impairment, lesion size, lesion side, and lateralization of language functions influence non-verbal functions.Methods: We recruited 8 patients with well-controlled epilepsies (9–26 years), 15 patients without epilepsies (8–23 years), and 23 healthy controls (8–27 years). All underwent the Test of Non-verbal Intelligence, a motor-independent test, which excludes biased results due to motor impairment. Language lateralization was determined with functional MRI, lesion size with MRI-based volumetry, and hand motor impairment with the Jebson-Taylor Hand Function-Test.Results: Patients with epilepsies showed significantly impaired non-verbal intelligence [Md = 89.5, interquartile range (IQR) = 13.5] compared with controls (Md = 103, IQR = 17). In contrast, patients without epilepsies (Md = 97, IQR = 15.0) performed within the range of typically developing children. A multiple regression analysis revealed only epilepsy as a significant risk factor for impaired non-verbal functions.Conclusion: In patients with unilateral perinatal strokes without epilepsies, the neuroplastic potential of one healthy hemisphere is able to support the development of normal non-verbal cognitive abilities, regardless of lesion size, lesion side, or language lateralization. In contrast, epilepsy substantially reduces this neuroplastic potential; even seizure-free patients exhibit below-average non-verbal cognitive functions.