We show that employing localized surface plasmon resonators to probe environmental media will always lead to dissimilar optical sensitivities to permittivity and permeability. We find that while the permittivity sensitivities of diverse plasmonic structures display a geometry-independent universal scaling relation, the permeability sensitivities are highly dependent on the metals' geometries and resonant modes. Similar results are also found in mixed real/spoof localized surface plasmon resonators, and the phenomena can be universally scaled to the normalized effective plasmon frequencies. Importantly, the results put a fundamental constraint for all plasmonic-assisted nonlinear magneto-optical phenomena, including the Faraday effect, magneto-optical Kerr effect, and Cotton-Mouton effect.