The cortical microcircuit is built with recurrent excitatory connections, and it has long been suggested that the purpose of this design is to enable intrinsically driven reverberating activity. To understand the dynamics of neocortical intrinsic activity better, we performed two-photon calcium imaging of populations of neurons from the primary visual cortex of awake mice during visual stimulation and spontaneous activity. In both conditions, cortical activity is dominated by coactive groups of neurons, forming ensembles whose activation cannot be explained by the independent firing properties of their contributing neurons, considered in isolation. Moreover, individual neurons flexibly join multiple ensembles, vastly expanding the encoding potential of the circuit. Intriguingly, the same coactive ensembles can repeat spontaneously and in response to visual stimuli, indicating that stimulus-evoked responses arise from activating these intrinsic building blocks. Although the spatial properties of stimulus-driven and spontaneous ensembles are similar, spontaneous ensembles are active at random intervals, whereas visually evoked ensembles are time-locked to stimuli. We conclude that neuronal ensembles, built by the coactivation of flexible groups of neurons, are emergent functional units of cortical activity and propose that visual stimuli recruit intrinsically generated ensembles to represent visual attributes.