Magnetorheological elastomers (MREs) are a class of smart materials with rubber-like qualities, demonstrating revertible magnetic field-dependent viscoelastic properties, which makes them an ideal candidate for development of the next generation of adaptive vibration absorbers. This research study aims at the development of a finite element model using microscale representative volume element (RVE) approach to predict the field-dependent shear behavior of MREs. MREs with different elastomeric matrices, including silicone rubber Ecoflex 30 and Ecoflex 50, and carbonyl iron particles (CIPs) have been considered as magnetic particles. The stress–strain characteristic of the pure silicon rubbers was evaluated experimentally to formulate the nonlinear Ogden strain energy function to describe hyper-elastic behavior of the rubbery matrix. The obtained mechanical and magnetic properties of the matrix and inclusions were integrated into COMSOL Multiphysics to develop the RVE for the MREs, in 2D and 3D configurations, with CIP volume fraction varying from 5% to 40%. Periodic boundary condition (PBC) was imposed on the RVE boundaries, while undergoing shear deformation subjected to magnetic flux densities of 0–0.4 T. Comparing the results from 2D and 3D modeling of isotropic MRE-RVE with the experimental results from the literature suggests that the 3D MRE-RVE can be effectively used to accurately predict the influence of varying factors including matrix type, volume fraction of magnetic particles, and applied magnetic field on the mechanical behavior of MREs.