Background Classical galactosemia (CG) is due to a severe deficiency of the galactose-1-phosphate uridyl-transferase (GALT), the main enzyme of galactose metabolism. Even early introduction of galactose-restricted diet fails to prevent long-term complications, including cognitive impairment, neurological and psychiatric problems, osteoporosis, premature ovarian failure and infertility. Detailed neuropsychological phenotyping is needed in order to better understand the relevant neurodevelopmental deficiencies and to develop effective treatment strategies. Aim To define specifically and significantly impaired neuropsychological traits in adult CG patients of the Swiss cohort. Methods Prospective cohort study. 22 CG patients, with confirmed genotype and low GALT activity, and 15 controls completed a computer-based neuropsychological test battery (CANTAB). Additionally, broad IQ evaluation was made for the CG patients. Results In most outcome measures of the CANTAB tasks, CG patients performed significantly worse than controls. The deficits in CG patients were most prominent in tasks that involve rapid visual information processing and facial emotion recognition. Conclusion CG patients have specific cognitive problems such as impaired visual information processing and facial emotion recognition. The deficits in facial emotion recognition have not been described before and could help explain difficulties in social interactions often experienced by patients with CG. Electronic supplementary material The online version of this article (10.1186/s13023-019-0999-3) contains supplementary material, which is available to authorized users.