Preventing femoral fractures is an important goal in osteoporosis research. In order to evaluate a person's fracture risk and to quantify response to treatment, bone competence is best assessed by bone strength. Finite-element (FE) modelling based on medical imaging is considered a very promising technique for the assessment of in vivo femoral bone strength. Over the past decades, a number of different FE models have been presented focusing on the effect of several methodological aspects, such as mesh type, material properties and loading conditions, on the precision and accuracy of these models. In this paper, a review of this work is presented. We conclude that moderate to good predictions can be made, especially when the models are tuned to specific loading scenarios. However, there is room for improvement when multiple loading conditions need to be evaluated. We hypothesize that including anisotropic material properties is the first target. As a proof of the concept, we demonstrate that the main orientation of the femoral bone structure can be calculated from clinical computed tomography scans. We hypothesize that this structural information can be used to estimate the anisotropic bone material properties, and that in the future this could potentially lead to a greater predictive value of FE models for femoral bone strength.