With the recently developed single-shot time-stretch dispersive Fourier transform technique, we investigate the buildup process of an all-polarization-maintaining soliton mode-locked fiber laser. Considering the multi-pulse competitions and the evolution of the survived pulse, we find an optimal range of intra-cavity energy for self-starting related to the saturation energy of the employed saturable absorber. Under the conditions, one dominant pulse can build up quickly against the others, and it finally drives to single-pulse operation. The conclusions drawn here hold for other soliton mode-locked lasers.