Bone mineral density, a bone matrix parameter frequently used to predict fracture risk, is not the only one to affect bone fragility. Other factors, including the extracellular matrix (ECM) composition and microarchitecture, are of paramount relevance in this process. The bone ECM is a noncellular three-dimensional structure secreted by cells into the extracellular space, which comprises inorganic and organic compounds. The main inorganic components of the ECM are calcium-deficient apatite and trace elements, while the organic ECM consists of collagen type I and noncollagenous proteins. Bone ECM dynamically interacts with osteoblasts and osteoclasts to regulate the formation of new bone during regeneration. Thus, the composition and structure of inorganic and organic bone matrix may directly affect bone quality. Moreover, proteins that compose ECM, beyond their structural role have other crucial biological functions, thanks to their ability to bind multiple interacting partners like other ECM proteins, growth factors, signal receptors and adhesion molecules. Thus, ECM proteins provide a complex network of biochemical and physiological signals. Herein, we summarize different ECM factors that are essential to bone strength besides, discussing how these parameters are altered in pathological conditions related with bone fragility.