The anisotropy of granular geomaterials is sensitive to their fabric, which exhibits anisotropic mechanical properties as a function of deposition history, microscopic fabric, and loading paths. Here, a new fabric-enriched continuum breakage model is proposed to examine the relation between elastic and inelastic anisotropy in granular materials and cemented granular materials. A microstructure model is first implemented in the framework of fabric-enriched continuum breakage mechanics (F-CBM), where the anisotropic behaviour prior to yielding is introduced through a symmetric second-order fabric tensor embedded in the expression of the elastic energy potential. The anisotropic strain energy storage prior to grain crushing leads to the rotation and distortion of the yield surface of cemented granular materials. Parametric analyses are performed to assess the overall capability of the model to characterize the anisotropic inelastic processes in cemented granular. It is shown that the proposed model can accurately predict the strong correlation between anisotropic elasticity and breakage-damage processes in cemented granular materials. When damage involving the skeleton is the dominant inelastic process, the size of the elastic domain contracts and the material exhibits augmented brittleness with the disintegration of cement. While breakage processes are predicted to dominate the response of lightly cemented granular materials resulting in hardening behaviour. This work can be further extended to dynamically capture the anisotropic response of cemented granular materials with water-sensitive mineral constituents by accounting for the evolution of microstructural anisotropy.