Iron-based soft magnetic composites (SMCs) are the key components of high-frequency electromagnetic systems. Selecting a suitable insulating oxide layer and ensuring the integrity and homogeneity of the heterogeneous core–shell structure of SMCs are essential for optimizing their magnetic properties. In this study, four types of SMCs—Fe-Si-Cr/ZrO2, Fe-Si-Cr/TiO2, Fe-Si-Cr/MgO, and Fe-Si-Cr/CaO—were prepared via ball milling, followed by hot-press sintering. The differences between the microscopic morphologies and magnetic fproperties of the Fe-Si-Cr/AOx SMCs prepared using four different metal oxides were investigated. ZrO2, TiO2, MgO, and CaO were successfully coated on the surface of the Fe-Si-Cr alloy powders through ball milling, forming a heterogeneous Fe-Si-Cr/AOx core–shell structure with the Fe-Si-Cr alloy powder as the core and the metal oxide as the shell. ZrO2 is relatively hard and less prone to breakage and refinement during ball milling, resulting in a lower degree of agglomeration on the surface of the composites and prevention of peeling and collapse during hot-press sintering. When ZrO2 was used as the insulation layer, the magnetic dilution effect was minimized, resulting in the highest resistivity (4.2 mΩ·cm), lowest total loss (580.8 kW/m3 for P10mt/100kHz), and lowest eddy current loss (470.0 kW/m3 for Pec 10mt/100kHz), while the permeability stabilized earlier at lower frequencies.