Time-frequency analysis always plays an important role in machine health monitoring owing to its advantage in extracting the fault information contained in non-stationary signal. In this paper, we present a novel technique to detect and diagnose the rolling bearing faults based on high-order synchrosqueezing transform (FSSTH) and detrended fluctuation analysis (DFA). With this method, the high-order synchrosqueezing transform is first utilized to decompose the vibration signal into an ensemble of oscillatory components termed as intrinsic mode functions (IMFs). Meanwhile, an empirical equation, which is based on the DFA, is introduced to adaptively determine the number of IMFs from FSSTH. Then, a timefrequency representation originated from the decomposed modes or corresponding envelopes is exhibited in order to identify the fault characteristic frequencies related to rolling bearing. Experiments are carried out using both simulated signal and real ones from Case Western Reserve University. Results show that the proposed method is more effective for the detection of fault characteristic frequencies compared with the traditional synchrosqueezing transform (SST) based fault diagnosis algorithm, which renders this technique is promising for machine fault diagnosis. INDEX TERMS Fault diagnosis, rolling bearing, time-frequency analysis, high-order synchrosqueezing transform, detrended fluctuation analysis.