Due to the fact that wave propagation has superior penetration at low frequencies, its attenuation in a limited range is a challenge. To solve this issue, a novel impedance modulating structure based on magnetorheological (MR) fluid is proposed, and the vibration transmissibility of the proposed structure is investigated. The modulation of impedance is realized by controlling the applied magnetic field, which can result in liquid-solid conversion of MR fluid. The wave equation in MR fluid is established based on the Biot theory. Subsequently, the wave equation is solved and the expression of vibration level difference (VLD) is obtained. Furthermore, the vibration transmissibility of different modulation parameters of impedance is analyzed. Finally, experiments are conducted to evaluate the variation in the vibration transmissibility depending on the modulation parameters of impedance. It is demonstrated that the modulation of MR fluid impedance is validly showing greater attenuation for wave propagation than that without modulation. Due to its great structural variability and adjustability parameter, the proposed structure has excellent controllability in vibration transference, and it is promising for further applications in isolating vibration at the platform of the large power systems.