In this study, we developed a data-driven frequency domain analysis method for rotor systems using the NARX (Nonlinear Auto-Regressive with eXternal input) model established by system vibration signals. We propose a model-based index of fault features calculated in a multi-frequency range to facilitate condition monitoring of rotor systems. Four steps are included in the proposed method. Firstly, displacement vibration signals are collected at multiple monitored rotating speeds. Secondly, the collected signals are processed as output data and the corresponding input data is generated. Then, NARX models are developed with input and output data to characterize the rotor system. Finally, the NRSF (Nonlinear Response Spectrum Function)-based nonlinear fault index is calculated and compared to the healthy condition. An experimental application to the misaligned rotor system is also demonstrated to verify its effectiveness. Our results indicate that the value of the index directly reflects the severity of the misaligned fault.