Gears are essential machine elements that are exposed to heavy loads. In some cases, gearboxes are critical elements since they serve as machine drivers that must operate almost every day for a more extended period, such as years or even tens of years. Any interruption due to gear failures can cause significant losses, and therefore it is necessary to have a monitoring system that will ensure proper operation. Tooth surface damage is a common occurrence in operating gears. One of the most common types of damage to teeth surfaces is pitting. It is necessary for normal gear operations to regularly determine the occurrence and span of a damaged tooth surface caused by pitting. In this paper, we propose a machine vision system as part of the inspection process for detecting pitting and monitoring its progression. The implemented inspection system uses a faster R-CNN network to identify and position pitting on a specific tooth, which enables monitoring. Prediction confidence values of pitting damage detection are between 99.5–99.9%, while prediction confidence values for teeth recognized as crucial for monitoring are between 97–99%.