To accurately measure the surrounding refractive index (SRI), an all-fiber microstructure multi-parameters optical sensor based on Mach–Zehnder interferometer (MZI) has been designed. A homemade elliptical multilayer-core fiber (EMCF), in which only two modes could be propagated, is used in this sensor. This sensor has a sandwich structure of EMCF-SMF-EMCF (ESE). The sensitivities of RI, temperature, and strain are analyzed practically, and different from each other which would provide a basis for restraining cross-sensitivity of sensor and improving measuring precision. According to the numerical simulation, the relationship between the guided mode and relevant excitation coefficient is shown, and the sensing characteristic of the interference spectrum is well expatiated. Maximum sensitivities of ∼31.83 nm per refractive index unit (RIU), ∼69 pm/°C, and ∼2.06 pm/µε have been experimentally achieved. The monitoring system is promoted by the fact that the resonance dips have their individual sensitivities, and the standard matrix inversion method is used for simultaneous three parameters determination. Consequently, this fiber sensor could ensure the completion of accurate SRI measurements with temperature and strain decoupled.