Music and language are two fundamental forms of human communication. Many studies examine the development of music‐ and language‐specific knowledge, but few studies compare how listeners know they are listening to music or language. Although we readily differentiate these domains, how we distinguish music and language—and especially speech and song— is not obvious. In two studies, we asked how listeners categorize speech and song. Study 1 used online survey data to illustrate that 4‐ to 17‐year‐olds and adults have verbalizable distinctions for speech and song. At all ages, listeners described speech and song differences based on acoustic features, but compared with older children, 4‐ to 7‐year‐olds more often used volume to describe differences, suggesting that they are still learning to identify the features most useful for differentiating speech from song. Study 2 used a perceptual categorization task to demonstrate that 4–8‐year‐olds and adults readily categorize speech and song, but this ability improves with age especially for identifying song. Despite generally rating song as more speech‐like, 4‐ and 6‐year‐olds rated ambiguous speech–song stimuli as more song‐like than 8‐year‐olds and adults. Four acoustic features predicted song ratings: F0 instability, utterance duration, harmonicity, and spectral flux. However, 4‐ and 6‐year‐olds’ song ratings were better predicted by F0 instability than by harmonicity and utterance duration. These studies characterize how children develop conceptual and perceptual understandings of speech and song and suggest that children under age 8 are still learning what features are important for categorizing utterances as speech or song.Research Highlights Children and adults conceptually and perceptually categorize speech and song from age 4. Listeners use F0 instability, harmonicity, spectral flux, and utterance duration to determine whether vocal stimuli sound like song. Acoustic cue weighting changes with age, becoming adult‐like at age 8 for perceptual categorization and at age 12 for conceptual differentiation. Young children are still learning to categorize speech and song, which leaves open the possibility that music‐ and language‐specific skills are not so domain‐specific.