To identify the damage within retaining wall structures, the Hilbert–Huang Transforms of the impulse response function and virtual impulse response function were performed. The Hilbert marginal energy ratio spectrums of the impulse response function and virtual impulse response function were acquired. To reflect damage information effectively, those bands with stronger damage sensitivity were extracted via the threshold value ε0. Then, the Hilbert feature bands, which were more sensitive to damage within retaining walls, were selected by considering the contribution of the residual band to the damage identification. Based on the feature bands, the Hilbert damage feature vector, which reflects the variations of Hilbert marginal energy ratio caused by damage, was created. Based on the damage feature vector, two damage identification indexes (the energy ration standard deviation and Energy Ration Standard Deviation), which were based on the impulse response function and virtual impulse response function, respectively, were proposed to identify damage within retaining walls. To investigate the validity of the damage indexes, vibration tests on a pile plate retaining wall were done. The test results show that the damage feature vector is a zero vector or the value of damage index is zero when the wall is undamaged. The damage feature vector is a nonzero vector or the value of the damage index is more than zero when the wall is damaged. Thus, the damage state of the wall can be detected sensitively via the damage feature vector or damage indexes. Partial damage causes greater fluctuation of trend surface of the damage index. The location of partial damage can be diagnosed validly via the coordinate of peak value in the trend surface. The quantitative relationship formula between the damage index and damage intensity is established. The damage intensity of the wall can be calculated reversely, when the damage index is available. Either the energy ration standard deviation or Energy Ration Standard Deviation can be used to detect the damage state, diagnose the damage location, and identify the damage intensity. In comparison with the energy ration standard deviation, the stability and damage sensitivity of the Energy Ration Standard Deviation is much better.