Under an external magnetic field, when circularly polarized light was transmitted through binary ferrofluids based on strongly magnetic γ-Fe2O3 and weakly magnetic ZnFe2O4 nanoparticles, the birefringence Δn and dichroism Δk arising from the chains of γ-Fe2O3 particles system were modulated and decreased by the unchained ZnFe2O4 particles. In our experiments, we used two types of ZnFe2O4 nanoparticles: one consisted of ZnFe2O4(1) particles with higher moments, and the other consisted of ZnFe2O4(2) particles with lower moments. Comparing the birefringence and dichroism of the γ-Fe2O3–ZnFe2O4(1) and γ-Fe2O3–ZnFe2O4(2) binary ferrofluids, it was found that the modulating action of the ZnFe2O4(2) particle system with lower moments was larger than that of the ZnFe2O4(1) particle system with higher moments. Using a model for a bi-dispersed system based on chained and unchained particles, the behavior of the modulating action was explained by an additional effective relative magnetic permeability, which depends on the background of the unchained ZnFe2O4 particles for the chained γ-Fe2O3 particles and a field-induced demixing phase transition. These results showed that for binary ferrofluids based on both strong and weak magnetic nanoparticles, the power of the modulation of the magneto-optical effects depends on the difference in magnetization between the particles