Vertebrates develop an olfactory system that detects odorants and pheromones through their interaction with specialized cell surface receptors on olfactory sensory neurons. During development, the olfactory system forms from the olfactory placodes, specialized areas of the anterior ectoderm that share cellular and molecular properties with placodes involved in the development of other cranial senses. The early-diverging chordate lineages amphioxus, tunicates, lampreys and hagfishes give insight into how this system evolved. Here, we review olfactory system development and cell types in these lineages alongside chemosensory receptor gene evolution, integrating these data into a description of how the vertebrate olfactory system evolved. Some olfactory system cell types predate the vertebrates, as do some of the mechanisms specifying placodes, and it is likely these two were already connected in the common ancestor of vertebrates and tunicates. In stem vertebrates, this evolved into an organ system integrating additional tissues and morphogenetic processes defining distinct olfactory and adenohypophyseal components, followed by splitting of the ancestral placode to produce the characteristic paired olfactory organs of most modern vertebrates.