Children learn their first number words gradually over the course of many months, which is surprising given their ability to discriminate small numerosities. One potential explanation for this is that children are sensitive to the numerical features of stimuli, but don’t consider exact cardinality as a primary hypothesis for novel word meanings. To test this, we trained 144 children on a number word they hadn’t yet learned, and contrasted this with a condition in which they were merely required to attend to number to identify the word’s referent, without encoding number as its meaning. In the first condition, children were trained to find a “giraffe with three spots.” In the second condition, children were instead trained to find “Mr. Three”, which also named a giraffe with three spots. In both conditions, children had to attend to number to identify the target giraffe, but, because proper nouns refer to individuals rather than their properties, the second condition did not require children to encode number as the meaning of the expression. We found that children were significantly better at identifying the giraffe when it had been labeled with the proper noun than with the number word. This finding contrasted with a second experiment involving color words, in which children (n = 56) were equally successful with a proper noun (“Mr. Purple”) and an adjective (“the giraffe with purple spots”). Together, these findings suggest that, for number, but not for color, children’s difficulty acquiring new words cannot be solely attributed to problems with attention or perception, but instead may be due to difficulty selecting the correct meaning from their hypothesis space for learning unknown words.