The creep and stress relaxation behaviors of coal are common in coal mining. The unified constitutive model is suitable to describe and predict both the creep and relaxation evolution characteristics of rocks. The generalized Kelvin model is the core element for traditional and improved component models to reflect both the nonlinear creep and relaxation. In this paper, an improved core damage model, which could both reflect the creep and stress relaxation in relation to the damage evolution, was established based on a comparison of the traditional and improved component models, and the responding constitutive equations (creep and stress relaxation equation) at constant stress/strain were deduced. Then, the core damage model was validated to the uniaxial compressive multistage creep and stress relaxation test results of coal, showing that the model curves had great accordance with the experimental data. Moreover, the model comparisons on accuracy, parameter meaning, and popularization among the core damage model, hardening-damage model, and the fractional derivative model were further discussed. The results showed that the parameters in the core damage model had clear and brief physical significances. The core damage model was also popularized to depict the time-dependent behaviors of other rocks, showing great accuracy.