Children with mathematical learning difficulties differ from their peers not only in terms of their mathematics-specific competencies, but also in terms of other cross-curricular areas of competence. In many of these areas, they make more mistakes and need more time. Numerous studies show that they also make more mistakes in working memory tasks. However, there is little research on whether they also need more time in working memory tasks. The present study addresses this question. Methodologically, our study is aligned to peer studies that are included in a current meta-analysis. Our results from over 400 first graders reveal that children with mathematical learning difficulties do not only make more mistakes in working memory tasks, but also need significantly more time for these cognitive processes already. Our findings highlight how important it is to consider not only mathematics-specific competencies but also working memory skills and the time needed when diagnosing and supporting children in order to effectively implement individualized interventions and meet heterogeneous learning conditions in mathematics classroom in primary school.