A major restriction in predicting plant community response to future climate change is a lack of long‐term data needed to properly assess species and community response to climate and identify a baseline to detect climate anomalies. Here, we use a 106‐year dataset on a Sonoran Desert plant community to test the role of extreme temperature and precipitation anomalies on community dynamics at the decadal scale and over time. Additionally, we test the climate sensitivity of 39 desert plant species and whether sensitivity is associated with growth form, longevity, geographic range, or local dominance. We find that desert plant communities have shifted directionally over the 106 years, but the climate had little influence on this directional change primarily due to non‐linear shifts in precipitation anomalies. Decadal‐scale climate had the largest impact on species richness, species relative density, and total plant cover, explaining up to 26, 45, and 55% of the variance in each, respectively. Drought and the interaction between the frequency of freeze events and above‐average summer precipitation were among the most influential climate factors. Increased drought frequency and wetter periods with frequent freeze events led to larger reductions in total plant cover, species richness, and the relative densities of dominant subshrubs Ambrosia deltoidea and Encelia farinosa. Over 80% of the tested species were sensitive to climate, where sensitivity was related to a species' local dominance but not longevity, geographic range, or growth form. Dominant species appear to exhibit demographic buffering. They were more sensitive to drought but better able to take advantage of hot and wet conditions than subdominant species. Overall, our results suggest that while decadal‐scale climate variation substantially impacts these desert plant communities, directional change in temperature over the last century has had little impact due to the relative importance of precipitation and drought. With projections of increased drought in this region, we may see reductions in total vegetation cover and species richness due to the loss of dominant species, possibly through a breakdown in their ability to demographically buffer climatic variation, potentially changing community dynamics through a change in facilitative and competitive processes.This article is protected by copyright. All rights reserved.