Bone is an intriguingly complex material. It combines high strength, toughness and lightweight via an elaborate hierarchical structure. This structure results from a biologically driven self-assembly and self-organisation, and leads to different deformation mechanisms along the length scales. Characterising multiscale bone mechanics is fundamental to better understand these mechanisms including changes due to bone-related diseases. It also guides us in the design of new bio-inspired materials. A key-gap in understanding bone’s behaviour exists for its fundamental mechanical unit, the mineralised collagen fibre, a composite of organic collagen molecules and inorganic mineral nanocrystals. Here, we report an experimentally informed statistical elasto-plastic model to explain the fibre behaviour including the nanoscale interplay and load transfer with its main mechanical components. We utilise data from synchrotron nanoscale imaging, and combined micropillar compression and synchrotron X-ray scattering to develop the model. We see that a 10-15% micro- and nanomechanical heterogeneity in mechanical properties is essential to promote the ductile microscale behaviour preventing an abrupt overall failure even when individual fibrils have failed. We see that mineral particles take up 45% of strain compared to collagen molecules while interfibrillar shearing seems to enable the ductile post-yield behaviour. Our results suggest that a change in mineralisation and fibril-to-matrix interaction leads to different mechanical properties among mineralised tissues. Our model operates at crystalline-, molecular- and continuum-levels and sheds light on the micro- and nanoscale deformation of fibril-matrix reinforced composites.