In this paper, we design and demonstrate an all-fiber-sensitive refractive index (RI) sensor based on the Mach–Zehnder interferometer (MZI). It is constructed by splicing two no-core fibers (NCFs) and a photonic crystal fiber (PCF) between two single-mode fibers (SMFs) to obtain an SMF–NCF–PCF–NCF–SMF composite structure (SNPNS). A study of the effect of varying PCF lengths on the RI reveals that the shorter the length, the higher the sensitivity. The maximum RI sensitivity of 176.9 nm/RIU is attained within the RI range of 1.3365–1.3767 when the PCF length in the SNPNS structure is 3 cm. Meanwhile, the sensor exhibits a high stability in water, with an RSD of only 0.0019% for the interference trough over a duration of two hours. This proposed sensing structure offers the advantages of a large extinction ratio, small size, low temperature sensitivity, and simple fabrication, exhibiting a great potential in RI measurements.