An Ms 6.4 earthquake occurred in Yangbi County, Yunnan Province, on May 21, 2021. The magnitude is only 0.1 difference when compared with the 2014 Ms 6.5 earthquake in Ludian County, Yunnan Province, but the disasters show a big difference. Seismic signal includes the comprehensive information of source, propagation path, and site effect, which directly reflect the effect of the earthquake on the ground. The study mainly investigated the difference between seismic signals of two events in time domain and time-frequency-energy space based on variational mode decomposition and Hilbert transform to clarify the reasons for disaster differences. Most damaged buildings are column-and-tie timber structures with Earth walls with local collapse or vertical crack at the corner of the Earth wall in the Yangbi earthquake. Multistorey buildings are intact, and a few small landslides are triggered, whereas, in the Ludian earthquake, multistorey buildings with the frame structure and masonry structure occurred with serious X-type cracking, large interlayer displacement, and even complete collapse, and large landslides with length about 165 m are triggered. 53LLT and 53YBX are two strong motion stations in the Ludian earthquake and Yangbi earthquake, respectively, and are near the disaster investigation points. Peak ground acceleration in 53LLT is up to 949.2 cm/s2, which is larger than 720.3 cm/s2 in 53YBX, and the duration in 53LLT is longer than 53YBX. The instantaneous energy of 53YBX is concentrated within 0–10 Hz and 10–20 Hz, respectively; however, the instantaneous energy of 53LLT is concentrated at 5 Hz. Cumulative energy of 53YBX is mainly distributed at 0–5 Hz and 10–20 Hz, which is around the natural frequency of column-and-tie timber structure, while the energy of 53LLT is concentrated at 1.6 Hz and is much greater than other stations. Thus, large energy in high frequency caused the instantaneous and cumulative damage to short-period structures in Yangbi County and may cause the resonance of column-and-tie timber structure, and large energy in low frequency caused the instantaneous and cumulative damage to long-period structures of Longtou Village in Ludian earthquake. The result is meaningful to understanding the seismic disasters from the characteristics of seismic signals and can provide a reference for the seismic design of structures.