BaFe 12−x Tb x/4 Tm x/4 Ti x/2 O 19 (x = 0 − 2.5 in a step of 0.5) substituted barium ferrite nanoparticles were synthesized by a sol-gel method. The structural characteristics of ferrite were studied by X-ray diffraction, scanning electron microscopy, transmission electron microscopy, FTIR, and 57 Fe Mössbauer spectroscopic measurements. It was found that single-phase barium ferrite nanoparticles with a narrow size distribution were successfully fabricated. The substituted cations were occupied in 12k sites. Vibrating sample magnetometer (VSM) results reflect that with an increase in substitution contents, the coercivity increases, while saturation of magnetization decrease. The reflection loss measurements were carried out on substituted ferrite nanoparticles. The obtained results revealed that the incorporation of Tm-Tb-Ti cations could enhance reflection loss and bandwidth in 8-12 GHz band. The role of matching thickness on the shifts of frequency was also investigated.