As hard and brittle materials are widely used in the modern industry, it is essential to effectively achieve uniform material removal. In the present study, a contact stress prediction model for double-layer elastomer under uniform displacement load was proposed. It aims to predict the contact stress distribution based on the structure size and material parameters of the double-layer elastomer and optimize the relevant parameters to improve the uniformity of material removal. The functionally graded and composite structured lapping and polishing plate (FG/CS-LPP) was developed by mixing abrasive particles and rubber with different mass ratios to get different Young's moduli. Through optical microscope, the surface morphology was observed, and the mechanism of Young's modulus gradient change was demonstrated. After numerical simulation analysis, the effect of Young's modulus fluctuation on contact stress distribution trend was evaluated. As for the contact stress prediction model, an equivalent Young's modulus model and compensation function H(z) was introduced to simplify the analytical process. Further, the obtained equivalent equation was verified by simulations and experiments with an average percentage deviation of~2% and~6%, respectively. By fitting calculated and simulated values with a deviation of less than 5%, the compensation function H(z) was established. Finally, the contact stress prediction model was obtained and validated by simulations and experiments, the percentage deviation is less than 13%. Accordingly, the contact stress prediction model can provide a theoretical basis for FG/CS-LPP to achieve uniform material removal.