In-wheel system is indispensible in eco-friendly vehicles including hybrid and fuel cell vehicles in regard to fuel consumption and degree of freedom. This paper focuses on designing and optimizing an Interior Permanent Magnet Synchronous Motor (IPMSM) for the system with the goals of minimizing torque ripple and Total Harmonic Distortion (THD) of line-to-line Back Electro-Motive Force (BEMF) through response surface methodology and finite element analysis since it is rarely possible to induce the equations which express the relationships between the design parameters and the objective functions. In addition, the IPMSM is comprised of 8 poles and 48 slots considering vibration and noise order, which is strongly connected to the magnitudes of torque ripple and THD of line-to-line BEMF. Particularly, the figures of barriers installed on both sides of magnets in rotor play the main role in satisfying the targets. In conclusion, the results from finite element analysis are compared with those from experiment to prove the validity. Having a lower torque ripple and THD of line-to-line BEMF, the optimum model is anticipated to show a lower degree of vibration and noise while the electromagnetic performances such as average torque and output power are maintained at the same with the prototype.