To improve the imaging accuracy in solving inverse scattering problems with mixed boundary conditions, the authors propose an inverse scattering imaging method based on a physics‐inspired neural network. First, the contrast source inversion and the T‐matrix method are unified to establish a combined parameter model, in which the characteristics of perfect electric conductor and dielectric scatterers are represented by the T‐matrix coefficients t and the dielectric contrast respectively. Considering the influence of singularity of conductor contrast, the PEC region and the DIE contrast are reconstructed separately. Then, an alternative parameter updating neural network method, called APU‐Net, is proposed to update the contrast of dielectric scatterers and the T‐matrix of perfect electric conductor scatterers alternately. The experimental results demonstrate the strong performance of APU‐Net for inverse scattering imaging of PEC‐DIE hybrid targets as well as its improved generalisation ability over existing methods.