Gradient nanostructured (GNS) surface layer with a controllable martensite fraction has been synthesized on 316L austenitic stainless steel by means of surface mechanical rolling treatment (SMRT) with temperature being controlled. The mean grain size is in the nanometer scale in the near-surface layer and increases gradually with depth. In addition, the volume fraction of martensite decreases from ~85% to 0 in the near-surface layer while the SMRT temperature increases from room temperature to 175 °C. Fatigue experiments showed that the strain-controlled fatigue properties of the GNS samples are significantly enhanced at total strain amplitudes ≥0.5%, especially in those with a dual-phase surface layer of austenite and pre-formed martensite. Analyses on fatigue mechanisms illustrated that the GNS surface layer enhances the strength-ductility synergy and suppresses the formation of surface fatigue defects during fatigue. In addition, the dual-phase structure promotes the formation of martensite and stacking faults, further enhancing fatigue properties at high strain amplitudes.