Material informatics has emerged as a valuable research field in material science, providing solutions to previously unsolvable problems or accelerating deliverables. Fatigue failure, as a complex and non‐deterministic phenomenon, requires a probabilistic approach to assess the uncertainty of the fatigue strength prediction. This study compares various probabilistic data‐driven models for credible fatigue strength predictions for three distinct steel groups. The analysis considers data and model uncertainty, evaluating their impacts on predictive quality from engineering and data science perspectives. Results reveal that deep ensembles outperform other probabilistic models regarding negative log‐likelihood (NLL), while random forest exhibits the lowest root mean square error (RMSE). Notably, the prediction accuracy of case‐hardened steels is negatively affected by insufficient material properties definitions, while stainless steels demonstrate the best performance compared to other steel types.