The flux switching permanent magnet (FSPM) motor is a stator permanent magnet motor. The typical FSPM motor is doubly salient, which generates torque ripple and affects the rotation smoothness. In this paper, the influence of salient pole geometric sizes on motor torque performance is studied to reduce motor torque ripple and improve motor torque density. First, three indicators are defined to present the motor torque characters, including the average torque, torque ripple, and the volume of permanent magnet material usage. Then, the model of salient pole geometric sizes function to the three indicators is created. With this model, the optimal solution is obtained by the multi-objective numerical optimization. Based on the optimized results, the rotor shape is refined and analyzed as the supplementary to reduce the torque ripple further. Finally, the results show that the optimization procedure in this paper can effectively weaken the torque ripple, reduce the permanent magnet material usage, and improve the average output torque of the motor. Based on the optimized data, the prototype machine is built to verify the analysis results and prove the feasibility of this research.