As a nondestructive testing technology, ground penetrating radar (GPR) is widely used in the detection and recognition of high-speed railway subgrade diseases. Rough-surface direct wave suppression caused by locomotive vibration and subgrade irregularity is a challenge in the GPR signal process. Flat-surface assumption makes traditional method difficult to apply directly to the rough terrain environment. To remove rough-surface direct wave and improve the signal-to-noise ratio, a new adaptive algorithm is proposed in this paper. First, according to the characteristic of electromagnetic wave propagation, the echo model of GPR was constructed, and the composition of ground radar echo signal was analyzed. Next, the eigenvalue and eigenvector of the echo signal and ground direct wave were studied. With the increase of relative permittivity, the energy eigenvector of direct wave and echo signal converged to the same vector space. On this basis, a rough-surface direct wave suppression framework based on energy feature adaptive analysis was proposed. Finally, a numerical simulation and field experiment were carried out to verify the feasibility of the algorithm. Results showed that the algorithm effectively suppresses the rough ground direct wave, and the processing result preserves the target echo more completely. Without intervention, this method can be applied in automatic direct wave suppression, which will promote the development of automatic identification technology in the detection of railway subgrade diseases.