Bone diseases such as osteoporosis are one of the main causes of bone fracture and often result in hospitalization and long recovery periods. Researchers are aiming to develop new tools that consider the multiple determinants acting at the different scales of bone, and which can be used to clinically estimate patient-specific fracture risk and also assess the efficacy of new therapies. The main step towards this goal is a deep understanding of the bone organ, and is achieved by modeling the complexity of the structure and the high variability of the mechanical outcome. This review uses a hierarchical approach to evaluate bone mechanics at the macroscale, microscale, and nanoscale levels and the interactions between scales. The first section analyzes the experimental evidence of bone mechanics in the elastic and inelastic regions, microdamage generation, and post-yield toughening mechanisms from the organ level to the ultrastructural level. On the basis of these observations, the second section provides an overview of the constitutive models available to describe bone mechanics and predict patient-specific outcomes. Overall, the role of the hierarchical structure of bone and the interplay between each level is highlighted, and their effect is evaluated in terms of modeling biological variability and patient specificity.