In the design of materials with low lattice thermal conductivity, compounds with high density, low speed of sound, and complexity at either the atomic, nano-or microstructural level are preferred. The layered compound Mg 3 Sb 2 defies these prevailing paradigms, exhibiting lattice thermal conductivity comparable to PbTe and Bi 2 Te 3 , despite its low density and simple structure. The excellent thermoelectric performance (zT ∼ 1.5) in n-type Mg 3 Sb 2 has thus far been attributed to its multi-valley conduction band, while its anomalous thermal properties have been largely overlooked. To explain the origin of the low lattice thermal conductivity of Mg 3 Sb 2 , we have used both experimental methods and ab initio phonon calculations to investigate trends in the elasticity, thermal expansion and anharmonicity of AMg 2 Pn 2 Zintl compounds with A = Mg, Ca, Yb, and Pn = Sb and Bi. Phonon calculations within the quasiharmonic approximation reveal large mode Grüneisen parameters in Mg 3 Sb 2 compared with isostructural compounds, in particular in transverse acoustic modes involving shearing of adjacent anionic layers. Measurements of the elastic moduli and sound velocity as a function of temperature using resonant ultrasound spectroscopy provide a window into the softening of the acoustic branches at high temperature, confirming their exceptionally high anharmonicity. We attribute the anomalous thermal behavior of Mg 3 Sb 2 to the diminutive size of Mg, which may be too small for the octahedrally-coordinated site, leading to weak, unstable interlayer Mg-Sb bonding. This suggests more broadly that soft shear modes resulting from undersized cations provide a potential route to achieving low lattice thermal conductivity low-density, earth-abundant materials.