This article presents a control system for a cutting tool condition supervision, which recognises tool wear automatically during turning. We used an infrared camera for process control, which—unlike common cameras—captures the thermographic state, in addition to the visual state of the process. Despite challenging environmental conditions (e.g., hot chips) we protected the camera and placed it right up to the cutting knife, so that machining could be observed closely. During the experiment constant cutting conditions were set for the dry machining of workpiece (low alloy carbon steel 1.7225 or 42CrMo4). To build a dataset of over 9,000 images, we machined on a lathe with tool inserts of different wear levels. Using a convolutional neural network (CNN), we developed a model for tool wear and tool damage prediction. It determines the state of a cutting tool automatically (none, low, medium, high wear level), based on thermographic process data. The accuracy of classification was 99.55%, which affirms the adequacy of the proposed method. Such a system enables immediate action in the case of cutting tool wear or breakage, regardless of the operator’s knowledge and competence.