A contact model for rough surfaces based on the fractal theory is proposed in the present work. Firstly, the deformation of the material is divided into four stages: elastic deformation, the first elastoplastic deformation, the second elastoplastic deformation, and full plastic deformation. And the variation of material hardness is considered when analyzing the contact characteristics of a single asperity within the first and second elastoplastic deformation stages. Secondly, the size distribution function of contact spots at different frequency levels is derived. And the expressions of asperity critical frequency levels are rederived. Lastly, the feasibility and credibility of the proposed model are verified by comparison with other contact models and experimental data. The results show that when the variation of the material hardness is considered, the contact area of a single asperity in the first elastoplastic deformation stage becomes larger, while the contact area of a single asperity in the second elastoplastic deformation stage becomes smaller. Moreover, the critical asperity frequency levels of the rough surface are not constant, but the variables are related to the total real contact area of the rough surface and decrease as the real contact area increases. The proposed model is a modification and improvement of the existing fractal contact models, which can lead to a more accurate relationship between the contact load and the total real contact area of the rough surface.