Collected mechanical signals usually contain a number of noises, resulting in erroneous judgments of mechanical condition diagnosis. The mechanical signals, which are nonlinear or chaotic time series, have a high computational complexity and intrinsic broadband characteristic. This paper proposes a method of gear and bearing fault classification, based on the local subspace projection noise reduction and PE. A novel nonlinear projection noise reduction method, smooth orthogonal decomposition (SOD), is proposed to denoise the vibration signals of various operation conditions. SOD can decompose the reconstructed multiple strands to identify smooth local subspace. In the process of projection from a high dimension to a low dimension, a new weight matrix is put forward to achieve a better denoising effect. Afterwards, permutation entropy (PE) is applied in the detection of time sequence randomness and dynamic mutation behavior, which can effectively detect and amplify the variation of vibration signals. Hence PE can characterize the working conditions of gear and bearing under different conditions. The experimental results illustrate the effectiveness and superiority of the proposed approach. The theoretical derivations, numerical simulations and experimental studies, all confirm that the proposed approach based on the smooth local subspace projection method and PE, is promising in the field of the fault classification of rotary machinery.