Response variability is an essential and universal feature of sensory processing and behavior. It arises from fluctuations in the internal state of the brain, which modulate how sensory information is represented and transformed to guide behavioral actions. In part, brain state is shaped by recent network activity, fed back through recurrent connections to modulate neuronal excitability. However, the degree to which these interactions influence response variability and the spatial and temporal scales across which they operate, are poorly understood. Here, we combined population recordings and modeling to gain insights into how neuronal activity modulates network state and thereby impacts visually evoked activity and behavior. First, we performed cellular-resolution calcium imaging of the optic tectum to monitor ongoing activity, the pattern of which is both a cause and consequence of changes in network state. We developed a minimal network model incorporating fast, short range, recurrent excitation and long-lasting, activity-dependent suppression that reproduced a hallmark property of tectal activity – intermittent bursting. We next used the model to estimate the excitability state of tectal neurons based on recent activity history and found that this explained a portion of the trial-to-trial variability in visually evoked responses, as well as spatially selective response adaptation. Moreover, these dynamics also predicted behavioral trends such as selective habituation of visually evoked prey-catching. Overall, we demonstrate that a simple recurrent interaction motif can be used to estimate the effect of activity upon the incidental state of a neural network and account for experience-dependent effects on sensory encoding and visually guided behavior.