The analysis of vibration signals has been a very important technique for fault diagnosis and health management of rotating machinery. Classic fault diagnosis methods are mainly based on traditional signal features such as mean value, standard derivation, and kurtosis. Signals still contain abundant information which we did not fully take advantage of. In this paper, a new approach is proposed for rotating machinery fault diagnosis with feature extraction algorithm based on empirical mode decomposition (EMD) and convolutional neural network (CNN) techniques. The fundamental purpose of our newly proposed approach is to extract distinguishing features. Frequency spectrum of the signal obtained through fast Fourier transform process is trained in a designed CNN structure to extract compressed features with spatial information. To solve the nonstationary characteristic, we also apply EMD technique to the original vibration signals. EMD energy entropy is calculated using the first few intrinsic mode functions (IMFs) which contain more energy. With features extracted from both methods combined, classification models are trained for diagnosis. We carried out experiments with vibration data of 52 different categories under different machine conditions to test the validity of the approach, and the results indicate it is more accurate and reliable than previous approaches.