In order to guarantee machining quality and production efficiency and as well as to reduce downtime and cost, it is essential to receive information about the cutting tool condition and change it in time if necessary. To this end, tool condition monitoring systems are of great significance. Based on the data fusion technology of multi-sensor integration and the learning ability of deep residual network, a tool wear monitoring system is proposed in this paper. For data acquisition, multiple sensors are used to collect vibration, noise and acoustic emission signals in the machining process. For signal processing, the data fusion method of angular summation of matrices is used, so that the multi-source data are fused into two-dimensional pictures. Then, the feature extraction module of the proposed system uses the pictures as input and takes advantage of the deep residual convolutional network in extracting the deep features from the pictures, and finally the identification module completes the recognition of the tool wear type. To verify its feasibility, a testing bed is built on a CNC milling machine, and multi-sensor data are collected during the machining process of a simple workpiece with one of the two selected cutting tools each time. The proposed system is trained using the collected data and then is used to monitor the wear condition of a new cutting tool by collecting real-time data under similar condition. The results show that the accuracy of wear state recognition is as high as 90%, which verifies the feasibility of the proposed tool wear monitoring system.