Owing to its broadband absorption, ultrafast recovery time, and excellent saturable absorption feature, graphene has been recognized as one of the best candidates as a high-performance saturable absorber (SA). However, the low absorption efficiency and reduced modulation depth severely limit the application of graphene-based SA in ultrafast fiber lasers. In this paper, a single-layer graphene saturable absorber mirror (SG-SAM) was coated by a quarter-wave SiO2 passivated layer, and a significantly enhanced modulation depth and reduced saturation intensity were obtained simultaneously compared to the SG-SAM without the SiO2 coating layer. In addition, long-term operational stability was found in the device due to the excellent isolation and protection of the graphene absorption layer from the external environment by the SiO2 layer. The high performance of the SAM was further confirmed by the construction of a ring-cavity EDF laser generating mode-locked pulses with a central wavelength of 1563.7 nm, a repetition rate of 34.17 MHz, and a pulse width of 830 fs.