We designed and tested an all-fiber, high efficiency Yb-doped laser operating at 1088 nm with a single-mode-multimode-single-mode (SMS) structure. A larger-mode-area gain fiber of 1.5 m length, with 20/130 μm core/cladding diameters was used to increase the absorption, and a diffraction-limited Gaussian output beam was obtained from the single-mode output fiber. Using a 976 nm laser diode as the pump source, the laser generated an output power up to 38.5 W with a slope efficiency of 70%. The output beam qualities, with and without SMS structure, were compared and showed that the fiber laser with the SMS structure can achieve high gain, short fiber length, and excellent beam quality.