A three-dimensional nonlinear constitutive model of the amplitude, frequency, magnetic and temperature dependent mechanical property of isotropic magneto-sensitive (MS) rubber is developed. The main components of MS rubber are an elastomer matrix and magnetizable particles. When a magnetic field is applied, the modulus of MS rubber increases, which is known as the magnetic dependence of MS rubber. In addition to the magnetic dependence, there are frequency, amplitude and temperature dependencies of the dynamic modulus of MS rubber. A continuum mechanical framework-based constitutive model consisting of a fractional standard linear solid (SLS) element, an elastoplastic element and a magnetic stress term of MS rubber is developed to depict the mechanical behavior of MS rubber. The novelty is that the amplitude, frequency, magnetic and temperature dependent mechancial properties of MS rubber are integrated into a whole constitutive model under the continuum mechanics frame. Comparison between the simulation and measurement results shows that the fitting effect of the developed model is very good. Therefore, the constitutive model proposed enables the prediction of the mechanical properties of MS rubber under various operating conditions with a high accuracy, which will drive MS rubber’s application in engineering problems, especially in the area of MS rubber-based anti-vibration devices.