The rheological properties of magnetorheological elastomers are influenced by magnetically sensitive fillers and the elastomer matrix. The ability to respond to an external magnetic field is imparted by the fillers, while the load-bearing capability is determined by the matrix type. In this paper, the effect of matrix material on the properties of magnetorhological elastomer is explored experimentally. Carbonyl iron particle content is varied by 0%, 15% and 25% by volume to produce magnetorheological elastomer samples using natural rubber, silicone rubber and nitrile butadiene rubber matrices. Forced transmissibility test approach was employed to evaluate the field induced variations in the dynamic stiffness and loss factor of magnetorheological elastomers. The dynamic stiffness of nitrile butadiene rubber is the highest, while that of silicone rubber is the lowest. Addition of carbonyl iron particles significantly improves stiffness, although these gains depend on the properties of unfilled matrix. The addition of 25% by volume of carbonyl iron particle increased the dynamic stiffness of a silicone rubber matrix based magnetorheological elastomer by 67.78%, while the similar change in magnetorheological elastomer with nitrile butadiene rubber matrix was 38.58%. The field dependent response of magnetorheological elastomers is governed by the matrix and ferromagnetic filler concentration. These qualities are higher in magnetorheological elastomer with a low initial dynamic stiffness matrix and lower in magnetorheological elastomers with a stiffer matrix.