Fibroblasts play an important role in maintaining tissue integrity by secreting components of the extracellular matrix and initiating response to injury. Although the function of fibroblasts has been extensively studied in adults, the embryonic origin and diversification of different fibroblast subtypes during development remain largely unexplored. Using zebrafish as a model, we show that the sclerotome, a sub-compartment of the somite, is the embryonic source of multiple fibroblast populations, including tenocytes (tendon fibroblasts), blood vessel associated fibroblasts, and fin mesenchymal cells. High resolution imaging shows that different fibroblast subtypes occupy unique anatomical locations with distinct morphologies. Photoconversion-based cell lineage analysis reveals that sclerotome progenitors at different dorsal-ventral and anterior-posterior positions display distinct differentiation potentials. Single cell clonal analysis suggests that sclerotome progenitors are multipotent, and the fate of their daughter cells is biased by their migration paths and relative positions. Using a small molecule inhibitor, we show that BMP signaling is required for the development of fin mesenchymal cells in the peripheral fin fold. Together, our work demonstrates that the sclerotome contains multipotent progenitors that respond to local signals to generate a diverse population of tissue-resident fibroblasts.