The vertebral column, with the centra as its iteratively arranged building blocks, represents the anatomical key feature of the vertebrate phylum. In contrast to amniotes, where vertebrae are formed from chondrocytes and osteoblasts deriving from the segmentally organized neural crest or paraxial sclerotome, teleost vertebral column development is initiated by chordoblasts of the primarily unsegmented axial notochord, while sclerotomal cells only contribute to later steps of vertebrae formation. Yet, for both mammalian and teleostean model systems, unrestricted signaling by Bone Morphogenetic Proteins (BMPs) or retinoic acid (RA) has been reported to cause fusions of vertebral elements, while the interplay of the two signaling processes and their exact cellular targets remain largely unknown. Here, we address this interplay in zebrafish, identifying BMPs as potent and indispensable factors that, as formerly shown for RA, directly signal to notochord epithelial cells/chordoblasts to promote entpd5a expression and thereby metameric notochord sheath mineralization. In contrast to RA, however, which promotes sheath mineralization at the expense of further collagen secretion and sheath formation, BMP defines an earlier transitory stage of chordoblasts, characterized by sustained matrix production/col2a1 expression and concomitant matrix mineralization/entpd5a expression. BMP-RA epistasis analyses further indicate that RA can only affect chordoblasts and their further progression to merely mineralizing cells after they have received BMP signals to enter the transitory col2a1/entpd5a double-positive stage. This way, both signals ensure consecutively for proper mineralization of the notochord sheath within segmented sections along its anteroposterior axis. Our work sheds further light onto the molecular mechanisms that orchestrate early steps of vertebral column segmentation in teleosts. Similarities and differences to BMP’s working mechanisms during mammalian vertebral column formation and the pathomechanisms underlying human bone diseases such as Fibrodysplasia Ossificans Progressiva (FOP) caused by constitutively active BMP signaling are discussed.