Mineralization is a process of deposition of calcium phosphate crystals within a fibrous extracellular matrix (ECM). In mineralizing tissues, such as dentin, bone and hypertrophic cartilage, this process is initiated by a specific population of extracellular vesicles (EV), called matrix vesicles (MV). Although it has been proposed that MV are formed by shedding of the plasma membrane, the cellular and molecular mechanisms regulating formation of mineralization-competent MV are not fully elucidated. In these studies, 17IIA11, ST2, and MC3T3-E1 osteogenic cell lines were used to determine how formation of MV is regulated during initiation of the mineralization process. In addition, the molecular composition of MV secreted by 17IIA11 cells and exosomes from blood and B16-F10 melanoma cell line was compared to identify the molecular characteristics distinguishing MV from other EV. Western blot analyses demonstrated that MV released from 17IIA11 cells are characterized by high levels of proteins engaged in calcium and phosphate regulation, but do not express the exosomal markers CD81 and HSP70. Furthermore, we uncovered that the molecular composition of MV released by 17IIA11 cells changes upon exposure to the classical inducers of osteogenic differentiation, namely ascorbic acid and phosphate. Specifically, lysosomal proteins Lamp1 and Lamp2a were only detected in MV secreted by cells stimulated with osteogenic factors. Quantitative nanoparticle tracking analyses of MV secreted by osteogenic cells determined that standard osteogenic factors stimulate MV secretion and that phosphate is the main driver of their secretion. On the molecular level, phosphate-induced MV secretion is mediated through activation of extracellular signal-regulated kinases Erk1/2 and is accompanied by re-organization of filamentous actin. In summary, we determined that mineralization-competent MV are distinct from exosomes, and we identified a new role of phosphate in the process of ECM mineralization. These data provide novel insights into the mechanisms of MV formation during initiation of the mineralization process.