The composite cylindrical shell pressure structure is widely used for autonomous underwater vehicle (AUV). To analyze the critical buckling problem of variable stiffness (VS) composite pressure structure of AUV, a discrete finite element (DFE) method based on the curve fiber path function is developed in this work. A design and optimization method based on the radial basis function surrogate method is proposed to optimize the critical buckling pressure for a VS composite cylindrical shell. Both the DFE and surrogate methods are verified to be valid by comparison with the experimental data from the listed references. The effects of the geometric parameter and fiber angle on the critical buckling pressure are studied for different cylindrical shell cases. The results indicate that the proposed simulation model and optimization method are accurate and efficient for the buckling analysis and optimization of a VS composite cylindrical shell. Optimization result shows that the optimum critical buckling pressure for the VS cylindrical shell is improved and is 21.1% larger than that of the constant stiffness cylindrical shell under the same geometric and boundary condition.