The mechanism of oxide inclusions in fatigue crack initiation in the very-high cycle fatigue (VHCF) regime was clarified by subjecting bearing steels deoxidized by Al (Al-deoxidized steel) and Si (Si-deoxidized steel) to ultrasonic tension-compression fatigue tests (stress ratio, R = −1) and analyzing the characteristics of the detected inclusions. Results show that the main types of inclusions in Si-and Al-deoxidized steels are silicate and calcium aluminate, respectively. The content of calcium aluminate inclusions larger than 15 µm in Si-deoxidized steel is lower than that in Al-deoxidized steel, and the difference observed may be attributed to different inclusion generation processes during melting. Despite differences in their cleanliness and total oxygen contents, the Si-and Al-deoxidized steels show similar VHCF lives. The factors causing fatigue failure in these steels reveal distinct differences. Calcium aluminate inclusions are responsible for the cracks in Al-deoxidized steel. By comparison, most fatigue cracks in Si-deoxidized steel are triggered by the inhomogeneity of a steel matrix, which indicates that the damage mechanisms of the steel matrix can be a critical issue for this type of steel. A minor portion of the cracks in Si-deoxidized steel could be attributed to different types of inclusions. The mechanisms of fatigue fracture caused by calcium aluminate and silicate inclusions were further analyzed. Calcium aluminate inclusions first separate from the steel matrix and then trigger crack generation. Silicate inclusions and the steel matrix are closely combined in a fatigue process; thus, these inclusions have mild effects on the fatigue life of bearing steels. Si/Mn deoxidation is an effective method to produce high-quality bearing steel with a long fatigue life and good liquid steel fluidity.