Inductance is an important parameter for motor design and control, and the performance of the motor is closely related to the inductance parameter. To solve the problem of the complex magnetic circuit structure of search coil mutual inductance, this paper takes an 8-pole and 48-slot interior permanent magnet synchronous motor (IPMSM) as the object to carry out the relevant research on the formula of search coil mutual inductance, illustrates the method of calculating the mutual inductance of the search coil, and compares and verifies the proposed method through finite element analysis and the actual measurement results. By setting up a procedure for calculating the magnetic flux of the search coil, the magnetic flux of each sub-coil of the search coil is analyzed and calculated. The mutual inductance of the search coil is calculated by summing up the magnetic chains of different sub-coils between the phases of the search coil. Lastly, the finite element simulation of the permanent magnet synchronous motor (PMSM) with the placement of the search coil is carried out, the inductance of the search coil in the experimental prototype is measured practically, and the finite element simulation results of the mutual inductance of the search coil are compared with the actual measurement results, which proves the correctness of the theoretical analysis.