Mathematics skills are associated with future employment, wellbeing and quality of life. However, many adults and children fail to learn the mathematics skills they require. To improve this situation, we need to have a better understanding of the processes of learning and performing mathematics. Over the past two decades there has been a substantial growth in psychological research focusing on mathematics. However, to make further progress we need to pay greater attention to the nature of, and multiple elements involved in, mathematical cognition. Mathematics is not a single construct; rather, overall mathematics achievement is comprised of proficiency with specific components of mathematics (e.g., number fact knowledge, algebraic thinking), which in turn recruit basic mathematical processes (e.g., magnitude comparison, pattern recognition). General cognitive skills and different learning experiences influence the development of each component of mathematics as well as the links between them. Here I propose and provide evidence for a framework that structures how these components of mathematics fit together. This framework allows us to make sense of the proliferation of empirical findings concerning influences on mathematical cognition and can guide the questions we ask, identifying where we are missing both research evidence and models of specific mechanisms.