High comorbidity rates have been reported between mathematical learning disabilities (MD) and reading and spelling disabilities (RSD). Research has identified skills related to math, such as number sense (NS) and visuospatial working memory (visuospatial WM), as well as to literacy, such as phonological awareness (PA), rapid automatized naming (RAN) and verbal short-term memory (Verbal STM). In order to explain the high comorbidity rates between MD and RSD, 7–11-year-old children were assessed on a range of cognitive abilities related to literacy (PA, RAN, Verbal STM) and mathematical ability (visuospatial WM, NS). The group of children consisted of typically developing (TD) children (n = 32), children with MD (n = 26), children with RSD (n = 29), and combined MD and RSD (n = 43). It was hypothesized that, in line with the multiple deficit view on learning disorders, at least one unique predictor for both MD and RSD and a possible shared cognitive risk factor would be found to account for the comorbidity between the symptom dimensions literacy and math. Secondly, our hypotheses were that (a) a probabilistic multi-factorial risk factor model would provide a better fit to the data than a deterministic single risk factor model and (b) that a shared risk factor model would provide a better fit than the specific multi-factorial model. All our hypotheses were confirmed. NS and visuospatial WM were identified as unique cognitive predictors for MD, whereas PA and RAN were both associated with RSD. Also, a shared risk factor model with PA as a cognitive predictor for both RSD and MD fitted the data best, indicating that MD and RSD might co-occur due to a shared underlying deficit in phonological processing. Possible explanations are discussed in the context of sample selection and composition. This study shows that different cognitive factors play a role in mathematics and literacy, and that a phonological processing deficit might play a role in the occurrence of MD and RSD.