Baleen whales, or mysticetes, include the largest vertebrates to have ever evolved. Their gigantism, evolutionary success, and ecological diversity have been linked to filter feeding. Mysticetes filter feed using elaborate keratinous baleen plates, which grow from the palate and allow them to strain large quantities of prey out of the water. While the earliest mysticetes retained the adult, mineralized teeth present in ancestral whales, all species of living baleen whales lack teeth and instead possess baleen. The mechanism by which this evolutionary transformation took place remains unknown. We present four independent, but non-exclusive hypotheses for the origin of baleen. We evaluate the support for these hypotheses based on separate lines of evidence, including paleontological, molecular, and ontogenetic data. We suggest that the origin of baleen is decoupled from the loss of teeth, with a separate morphological and genetic basis. Moreover, we outline how new fossils and phylogenetic analyses may resolve current debates about morphological transformations in tooth loss and baleen origin across the phylogeny of stem and crown Mysticeti. Additional insights will likely arise from more detailed examination of developmental and biomechanical data, with sufficient ontogenetic and phylogenetic sampling.