This paper addresses the quantification of environmental variability of wavelet packet energy spectrum (WPES) extracted from the ambient dynamic responses of a suspension bridge using wavelet packet transform (WPT). The daily averaged WPES using multi-sample averaging technique are first obtained to eliminate the inherent randomness arising from the identification algorithm. Then the effect of temperature on the measured WPES is quantified using the seasonal correlation models. The traffic-induced and wind-induced variability are further quantitatively evaluated by establishing the traffic-WPES and wind-WPES correlation models. The analysis results reveal that temperature and inherent randomness are the critical sources causing WPES variability, and the WPES variability caused by wind speed and traffic loadings is negligible compared with temperature and inherent randomness. Considering seasonal correlation models of temperature-WPES can effectively eliminate the temperature effect and inherent randomness, it is suitable for structural damage warning of long-span bridges if future seasonal correlation models deviate from these normal models.