Fault diagnosis of rolling bearings is important for ensuring the safe operation of industrial machinery. How to effectively extract the fault features and select a classifier with high precision is the key to realizing the fault recognition of bearings. Accordingly, a new fault diagnosis method of rolling bearings based on improved fast spectral correlation and optimized random forest (i.e., particle swarm optimization-random forest (PSO-RF)) is proposed in this paper. The main contributions of this study are made from two aspects. One is that an improved fast spectral correlation approach was developed to extract the fault features of bearings and form the feature vector more effectively. The other is that an optimized random forest classifier was developed to achieve highly accurate identification by exploiting particle swarm optimization to select the best parameters of random forest (RF). In the presented method, improved fast spectral correlation was first utilized to analyze the raw vibration signal caused by a faulty bearing to obtain the enhanced envelope spectrum. Then, the amplitudes of the four characteristic cyclic frequencies (i.e., the rotating frequency, the characteristic frequency of outer-race fault, the characteristic frequency of inner-race fault, and the characteristic frequency of rolling element fault) exhibited in the enhanced envelope spectrum were selected to form the feature vector. Finally, the PSO-RF method was introduced for identifying and classifying bearing faults. The experimental investigations demonstrate the proposed method can accurately identify bearing faults and outperform other state-of-art techniques considered.