According to one of the most powerful paradigms explaining the meaning of the concept of natural number, natural numbers get a large part of their conceptual content from core cognitive abilities. Carey’s bootstrapping provides a model of the role of core cognition in the creation of mature mathematical concepts. In this paper, I conduct conceptual analyses of various theories within this paradigm, concluding that the theories based on the ability to subitize (i.e., to assess an exact quantity of the elements in a collection without counting them), or on the ability to approximate quantities (i.e., to assess an approximate quantity of the elements in a collection without counting them), or both, fail to provide a conceptual basis for bootstrapping the concept of an exact natural number. In particular, I argue that none of the existing theories explains one of the key characteristics of the natural number structure: the equidistances between successive elements of the natural numbers progression. I suggest that this regularity could be based on another innate cognitive ability, namely sensitivity to the regularity of rhythm. In the final section, I propose a new position within the core cognition paradigm, inspired by structuralist positions in philosophy of mathematics.