Aiming at the impact of heat generation and temperature rise on the driving performance of a permanent magnet (PM) motor, taking the PM in-wheel motor (IWM) for electric vehicles as an object, research is conducted into the temperature distribution of the electromagnetic–thermal effect and cooling structure optimization. Firstly, the electromagnetic–thermal coupling model considering electromagnetic harmonics is established using the subdomain model and Bertotti’s iron loss separation theory. Combined with the finite element (FE) simulation model established by Ansoft Maxwell software platform, the winding copper loss, stator core loss and PM eddy current loss under the action of complex magnetic flux are analyzed, and the transient temperature distribution of each component is obtained through coupling. Secondarily, the influence of the waterway structure parameters on the heat dissipation effect of the PM-IWM is analyzed by the thermal-fluid coupled relationship. On the basis, the optimization design of waterway structure parameters is carried out to improve the heat dissipation effect of the cooling system based on the proposed chaotic mapping ant colony algorithm with metropolis criterion. The comparison before and after optimization shows that the temperature of key components is significantly improved, the average convection heat transfer coefficient (CHTC) is increased by 23.57%, the peak temperature of stator is reduced from 95.47 °C to 82.73 °C, and the peak temperature of PM is decreased by 14.26%, thus the demagnetization risk in the PM is improved comprehensively. The research results can provide some theoretical and technical support for the structural optimization of water-cooled dissipation in the PM motor.