The rotate vector (RV) reducer has a complex structure and highly coupled internal components. Acoustic emission (AE) signal, which is more sensitive to a weak fault, is selected for fault diagnosis of the RV reducer. The high sampling frequency and big data are the challenges for AE signal store and analysis. This study combines compressed sensing (CS) and convolutional neural networks. As a result, data redundancy is significantly reduced while retaining most of the information, and the analysis efficiency is improved. Firstly, the time-domain AE signal was projected into the compression domain to obtain the compression signal; then, the wavelet packet decomposition in the compressed domain was performed to obtain the information of each frequency band. Next, the frequency band information was sent into the input layer of the multi-channel convolutional layer, and the energy pooling layer mines the energy characteristics of each frequency band. Finally, the softmax classifier was used to classify and predict different fault types of RV reducers. The self-fabricated RV reducer experimental platform was used to verify the proposed method. The experimental results show that the proposed method can effectively extract the fault features in the AE signal of the RV reducer, improve the efficiency of signal processing and analysis, and achieve the accurate classification of RV reducer faults.