The fault feature signal of rolling bearing can be characterized as the narrow-band signal with a specific resonance frequency. Therefore, resonance demodulation analysis is a powerful damage detection technique of bearings. In addition to the fault feature signal, the measured vibration signals carry various interference components, and these interference components become a serious obstacle of fault feature extraction. Variational mode extraction is a novel signal analysis method designed to retrieve a specific signal component from the composite signal. Variational mode extraction is founded on a similar basis as variational mode decomposition, while it shows better accuracy and higher efficiency compared with variational mode decomposition. In this study, variational mode extraction is introduced to the resonance demodulation analysis of bearing fault. As the results of variational mode extraction analysis are greatly influenced by the choice of two parameters, that is, the balancing factor α and the initial guess of center frequency ωd, an optimized variational mode extraction method is further developed. First, a new fault information evaluation index for measuring the richness of fault characteristics of the signal, termed ensemble impulsiveness and cyclostationarity, is formulated. Second, the ensemble impulsiveness and cyclostationarity is used as the fitness function of particle swarm optimization to automatically determine the optimal values of α and ωd. Finally, the validity of optimized variational mode extraction method is verified by simulated and experimental analysis, and the superiority of optimized variational mode extraction method is highlighted through comparison with two other advanced resonance demodulation analysis approaches, that is, the improved kurtogram and infogram. The analysis results indicate that optimized variational mode extraction method has a powerful capability of resonance demodulation analysis.