Tool steels are used in stamping, shearing processes, and as cutting tools due to their good mechanical properties. During their working cycle, steels are subject to aggressive conditions such as heat stress, fatigue, and wear. In this paper, three tool steels, namely X153CrMoV12, X37CrMoV5-1, and X45NiCrMo4 were selected against two types of bearing balls, ZrO2 and X46Cr1. All measurements were performed on a UMT TriboLab universal tribometric instrument under dry conditions. The main objective of the experiment was to analyze and compare tool steel wear in contact with two kinds of bearing balls with a diameter of 4.76 mm. This evaluation is focused on the hardness, surface roughness, and microstructure of all samples and on the impact of the input parameters on the resulting wear. All three types of tool steels were measured in the basic annealed state and, subsequently, in the state after hardening and tempering. Experimental results show that tool steels, belonging to high strength steels, can successfully represent wear resistant steels. The content of carbide elements, their size, and shape in the microstructure play an important role in the friction process and subsequent wear. Three types of loads were used and compared in the experiments 30, 60, and 90 N. Increasing the load results in significant degradation of the material on the sample surface. Lastly, the impact of hardness and roughness of materials on wear has also been proven. If abrasive wear occurs in the friction process, there is a greater degree of wear than that of adhesive wear. This is due to less abrasive particles, which behave like a cutting wedge and are subject to subsequent deformation strengthening due to the load increase, which adversely affects the further friction process. Analysis of the results showed that the ZrO2 ceramic ball showed significantly better wear values when compared to the X46Cr13 stainless steel ball. It also improves the values of the coefficient of friction with respect to the type of wear that occurs when the experimental materials and counterparts are in contact.