A new state-based elasto-plastic constitutive relationship along with the discrete element model is established to estimate the degradation of granular materials due to internal erosion. Four essential effects of internal erosion such as the force network damage and relaxation are proposed and then incorporated into the constitutive relationship to formulate internal erosion impacts on the mechanical behavior of granular materials. Most manifestations in the degradation of granular materials, such as reduction of peak strength and dilatancy are predicted by the modified constitutive relationship in good agreement with the discrete element method (DEM) simulation. In particular, the sudden reduction of stress for conspicuous mass erosion in a high stress state is captured by force network damage and the relaxation mechanism. It is concluded that the new modified constitutive relationship is a potential theory to describe the degradation of granular materials due to internal erosion and would be very useful, for instance, in the prediction and assessment of piping disaster risk during the flood season.