Several studies on long-term air pollution exposure and sleep have reported inconsistent results. Large-scale studies on short-term air pollution exposures and sleep have not been conducted. We investigated the associations of long- and short-term exposure to ambient air pollutants with sleep in a Chinese population based on over 1 million nights of sleep data from consumer wearable devices. Air pollution data including particulate matter (PM2.5, PM10), nitrogen dioxide (NO2), sulfur dioxide (SO2), carbon monoxide (CO), and ozone (O3) were collected from the Ministry of Ecology and Environment. Short-term exposure was defined as a moving average of the exposure level for different lag days from Lag0 to Lag0-6. A 365-day moving average of air pollution was regarded as long-term exposure. Sleep data were recorded using wearable devices from 2017 to 2019. The mixed-effects model was used to evaluate the associations. We observed that sleep parameters were associated with long-term exposure to all air pollutants. Higher levels of air pollutant concentrations were associated with longer total sleep and light sleep duration, shorter deep sleep duration, and decreases in wake after sleep onset (WASO), with stronger associations of exposures to NO2 and CO [a 1-interquartile range (IQR) increased NO2 (10.3 μg/m3) was associated with 8.7 min (95% CI: 8.08 to 9.32) longer sleep duration, a 1-IQR increased CO (0.3 mg/m3) was associated with 5.0 min (95% CI: − 5.13 to − 4.89) shorter deep sleep duration, 7.7 min (95% CI: 7.46 to 7.85) longer light sleep duration, and 0.5% (95% CI: − 0.5 to − 0.4%) lower proportion of WASO duration to total sleep]. The cumulative effect of short-term exposure on Lag0-6 is similar to long-term exposure but relatively less. Subgroup analyses indicated generally greater effects on individuals who were female, younger (< 45 years), slept longer (≥ 7 h), and during cold seasons, but the pattern of effects was mixed. We supplemented two additional types of stratified analyses to reduce repeated measures of outcomes and exposures while accounting for individual variation. The results were consistent with the overall results, proving the robustness of the overall results. In summary, both short- and long-term exposure to air pollution affect sleep, and the effects are comparable. Although people tend to have prolonged total sleep duration with increasing air pollutant concentrations, their sleep quality might remain poor because of the reduction in deep sleep.