Nonlinear vibration issues are of great importance in physics, mechanical structures and other engineering research. Vibration response, stability, and frequencies are the main components of a system's vibration check. Therefore, investigating the effect of different parameters in these sections can be an important step in the design process. In recent years, much research has been done on nonlinear vibrations, and analytical and numerical methods have been used to solve complex nonlinear equations. Equipment and the use of marine structure laboratories for the practical study of the dynamic behavior of vessels require large expenditures, so in order to advance computer science, several software's have been developed and used to model and perform dynamic analysis. In this paper, the mass of the shaft, the motor assembly and the impeller are each examined as a centralized mass and are modelled by ANSYS software. Then, by extracting vibrational equations along the shaft axis and multi-objective optimization with response surface method (RSM), the comparison of the results of this method with the work of other researchers shows that this method has a good accuracy.