Due to the mode mixing, empirical mode decomposition (EMD) cannot effectively decompose the vibration signal when the signal is intermittent and pulse interference caused by discontinuous vibration. The methods to solve mode mixing often use noise assistance, such as ensemble EMD (EEMD), complete EEMD (CEEMD), etc. These methods can effectively solve mode mixing, but they also have shortcomings. In EEMD, the added noises not only have residual effects and time-consuming. The drawback of CEEMD is that it is difficult to align during set averaging. In this paper, an improved EMD based on binary time scale (EMD-BTS) is proposed for the fault feature extraction of wheel–rail defect detection. Firstly, the generalized intrinsic mode function (GIMF) is defined based on the time-domain characteristics of non-stationary vibration signals. Then, to tackle the drawbacks of EMD which cannot effectively solve mode mixing caused by signal intermittence and pulse interference, the inherent mode is extracted in the EMD-BTS to decompose the raw signals into GIMFs. Finally, the false components generated by over decomposition are combined based on time-domain cross covariance. A simulation case and a actual case of vehicle bogie are utilized to verify the feasibility of the proposed EMD-BTS. The results indicate that the proposed approach exceeds other typical techniques in extracting intermittent fault features of wheel–rail defect detection.