Fluctuations in Surface Enhanced Raman Spectroscopy (SERS) signals are a hallmark of few-to-single molecule events. Recent experiments have shown these SERS intensity fluctuations (SIFs) to occur over an extremely wide range of time scales, from seconds to microseconds. While many mechanisms have been proposed, such as molecular diffusion or transient plasmonic hotspot generation, the underlying source of these fluctuations is likely to be a complex interplay of different effects. For example, SERS experiments done with fully coated, but dry, nanoparticles would have less movement of the probe molecule but may still have transient hotspot generation due to the mobility of metallic atoms. Alternatively, experiments done with low concentrations of a molecular probe in liquid might tend to see fluctuations caused by freely diffusing molecules that visit and become trapped within a static hotspot. In this paper we compare high-speed SIF activity in both dry and wet nanoparticle environments. By carefully analyzing the overall SERS signal fluctuations, such as the timing statistics, we propose a simple model that accounts for the contributions from the various mechanisms discussed above. These results provide a deeper physical understanding of the SERS effect and may promote further research into singlemolecule SERS, its experimental optimization, and potential applications.