Microstructural heterogeneities of an ASTM A128 grade C steel subjected to heat treatments and their effect on abrasive wear behavior were investigated. The heat-treatment process involved different austenization times at 1050 °C and quenching media. To characterize the effects of heat treatment on the material’s microstructure and mechanical behavior, two microscopy techniques were used: optical microscopy (OM), and scanning electron microscopy (SEM). The chemical composition of the material was obtained using X-ray fluorescence (XRF) optical emission spectrometry. The variation in carbide composition was evaluated using X-ray Energy Dispersive Spectroscopy (EDS). To characterize the mechanical behavior of the steel, hardness measurements and abrasive wear tests were performed after homogenization annealing and quenching treatments. The results showed that the heat-treated samples developed a heterogeneous microstructure, with the presence of austenitic grains and Martensite around the surface of the samples induced by decarburization in both the protected and unprotected specimens. The specimens with lower decarburization presented less formation of Martensite and precipitated carbides, resulting in lower hardness values and higher abrasion resistance (solution treatment at 1080 °C for 1 h + sand protection + brine quenching).