Tissues of the brain, especially white matter, are extremely heterogeneous—with constitutive responses varying spatially. In this paper, we implement a high-resolution Finite Element (FE) head model where heterogeneities of white matter structures are introduced through Magnetic Resonance Elastography (MRE) experiments. Displacement of white matter under shear wave excitation is captured and the material properties determined through an inversion algorithm are incorporated in the FE model via a two-term Ogden hyper-elastic material model. This approach is found to improve model predictions when compared to experimental results. In the first place, mechanical response in the cerebrum near stiff structures such as the corpus callosum and corona radiata are markedly different compared with a homogenized material model. Additionally, the heterogeneities introduce additional attenuation of the shear wave due to wave scattering within the cerebrum.