Electrophysiological responses of SCN neurons to light steps are well established, but responses to more natural modulations in irradiance have been much less studied. We address this deficit first by showing that variations in irradiance for human subjects are biased towards low temporal frequencies and small magnitudes. Using extracellular recordings we show that neurons in the mouse SCN are responsive to stimuli with these characteristics, tracking sinusoidal modulations in irradiance best at lower temporal frequencies and responding to abrupt changes in irradiance over a range of commonly encountered contrasts. The spectral sensitivity of these light adapted responses indicates that they are driven primarily by cones, but with melanopsin (and/or rods) contributing under more gradual changes. Higher frequency modulations in irradiance increased time averaged firing of SCN neurons (typically considered to encode background light intensity) modestly over that encountered during steady exposure, but did not have a detectable effect on the circadian phase resetting efficiency of light. Our findings highlight the SCN’s ability to encode naturalistic temporal modulations in irradiance, while revealing that the circadian system can effectively integrate such signals over time such that phase-resetting responses remain proportional to the mean light exposure.