A buckling defect will appear on the outer surface of the deformed ring during the constrained ring rolling (CRR) of an aluminum alloy thin-wall conical ring with inner high ribs (AATWCRIHR) if the geometrical dimension of the ribs does not match the wall thickness. To avoid the buckling defect, a quantitative method for characterizing the degree of the buckling defect is proposed using the area of the buckling profile. Then, an orthogonal experimental scheme was designed, taking the width of the middle rib, thickness of wall, and height of the middle rib as the design variables and defining the area of the buckling profile as the optimization objective. Subsequently, a quadratic polynomial response surface model was established by combining the optimization algorithm with the finite element method (FEM), and the geometrical dimension of the middle ribs of the deformed AATWCRIHR is optimized. Moreover, the optimal parameter combination to minimize the area of the buckling profile is obtained and verified using FE simulation. The results show that the AATWCRIHR after optimization does not generate the buckling defect during constrained ring rolling, and it is proven that the quantitative buckling defect representation method and the optimization design method based on the response surface model and the finite element simulation results are feasible for the constrained ring-rolling process of the AATWCRIHR.