Brain injuries are often characterized by diffusely distributed axonal and vascular damage invisible to medical imaging techniques. The spatial distribution of mechanical stresses and strains plays an important role, but is not sufficient to explain the diffuse distribution of brain lesions. It remains unclear how forces are transferred from the organ to the cell scale and why some cells are damaged while neighboring cells remain unaffected. To address this knowledge gap, we subjected histologically stained fresh human and porcine brain tissue specimens to compressive loading and simultaneously tracked cell and blood vessel displacements. Our experiments reveal different mechanisms of load transfer from the organ or tissue scale to single cells, axons, and blood vessels. Our results show that cell displacement fields are inhomogeneous at the interface between gray and white matter and in the vicinity of blood vessels—locally inducing significant deformations of individual cells. These insights have important implications to better understand injury mechanisms and highlight the importance of blood vessels for the local deformation of the brain’s cellular structure during loading.