We have experimentally observed ultrashort pulse in a simple, all-polarization-maintaining fiber, master oscillator and power amplifier laser. A semiconductor saturable mirror (SESAM) module is used as the mode locker. The ultrashort pulse exhibits perfect Gaussian spectrum without any fine structure. The 20 dB pulse spectral width is 28 nm, and the pulse duration directly output from the oscillator is 268.6 fs. Via a two-stage amplifier, the pulse spectrum is greatly broadened to 43 nm, and the pulse duration lengths to 0.9 ps. Moreover, after compression, a typical saddle-shaped spectrum appears and the spectral bandwidth broadens to 121 nm; the pulse duration is compressed to ~35 fs with a pulse peak power of 22.79 kW, which is high enough to stimulate a one-octave supercontinuum spectrum. Due to the polarization-maintaining structure and high reliability of SESAM, the whole system exhibits strong environmental stability. Experimental observation reveals that such a fiber laser with a very simple structure exhibits excellent performance, and could be applied in the area of optical frequency comb.