The study aims to evaluate the influence of surface modification of Fe powder on the magnetic behavior of soft magnetic compacts and composites that can possibly enhance their properties. The smoothing of ferromagnetic particle surfaces led to a decrease in the total energy loss as the most evident positive impact in all investigated classes (max. by 11% for small, 63–125 μm particle-based annealed Fe compacts, at max. induction 0.5 T and frequency 100 Hz) and to a partial increase in specific electrical resistivity (max. by 47% for small particle-based Fe@SiO2 composites) and resonant frequency (max. by 48% for large, 200–400 μm particle-based Fe@SiO2 composites) as well as partial decrease in coercivity (max. by 14% for small particle-based annealed Fe compacts). Removing surface irregularities negatively affected the maximum total permeability (max. drop by 28% for large particle-based Fe@SiO2 composites) due to increased inner demagnetizing fields. Applying the Bertotti theory for loss separation, we obtained parameters of loss components and assumed the domain structure using simultaneously active magnetic objects as predictors. The total loss decrease observed after the smoothing process originates from the significantly increased numbers of active magnetic objects, facilitating AC magnetization reversal so that domain wall displacements are accompanied by lower energy loss, manifested as a decrease in the excess loss component (max. by 61% for small particle-based Fe@SiO2 composites).