The crack initiation and short crack propagation in a martensitic spring steel were investigated by means of in-situ fatigue testing. Shot peened samples as well as untreated samples were exposed to uniaxial alternating stress to analyze the impact of compressive residual stresses. The early fatigue damage started in both sample conditions with the formation of slip bands, which subsequently served as crack initiation sites. Most of the slip bands and, correspondingly, most of the short fatigue cracks initiated at or close to prior austenite grain boundaries. The observed crack density of the emerging network of short cracks increased with the number of cycles and with increasing applied stress amplitudes. Furthermore, the prior austenite grain boundaries acted as obstacles to short crack propagation in both sample conditions. Compressive residual stresses enhanced the fatigue strength, and it is assumed that this beneficial effect was due to a delayed transition from short crack propagation to long crack propagation and a shift of the crack initiation site from the sample surface to the sample interior.