Bone replacement might have been practiced for centuries with various materials of natural origin, but had rarely met success until the late 19th century. Nowadays, many different bone substitutes can be used. They can be either derived from biological products such as demineralized bone matrix, platelet-rich plasma, hydroxyapatite, adjunction of growth factors (like bone morphogenetic protein) or synthetic such as calcium sulfate, tri-calcium phosphate ceramics, bioactive glasses, or polymer-based substitutes. All these substitutes are not suitable for every clinical use, and they have to be chosen selectively depending on their purpose. Thus, this review aims to highlight the principal characteristics of the most commonly used bone substitutes and to give some directions concerning their clinical use, as spine fusion, open-wedge tibial osteotomy, long bone fracture, oral and maxillofacial surgery, or periodontal treatments. However, the main limitations to bone substitutes use remain the management of large defects and the lack of vascularization in their central part, which is likely to appear following their utilization. In the field of bone tissue engineering, developing porous synthetic substitutes able to support a faster and a wider vascularization within their structure seems to be a promising way of research.