Belt conveyor is considered as a momentous component of modern coal mining transportation system, and thus it is an essential task to diagnose and monitor the damage of belt in real time and accurately. Based on the deep learning algorithm, this present study proposes a method of conveyor belt damage detection based on ADCN (Adaptive Deep Convolutional Network). A deep convolution network with unique adaptability is built to extract the different scale features of visible light image of conveyor belt damage, and the target is classified and located in the form of anchor boxes. A data set with data diversity is collected according to the actual working conditions of the conveyor belt. After training and regression, the ADCN model can perfectly capture and classify the damaged target in the video of the conveyor running. Compared with the SVM based method, the method based on ADCN can better meet the realtime and reliability requirements of belt damage detection, and it has the positioning function which SVM does not have.