The striatum features a distinct network characterized by a high degree of shared feedforward inhibition (FFI) from a mere 1% of fast-spiking interneurons (FSI). We investigate the potential roles of this extensively shared FFI in striatal function beyond inducing synchrony. Our findings reveal that FSIs increase the across-trial variability of striatal responses to cortical stimuli and, combined with recurrent inhibition, lead to a 'correlation attractor' of striatal activities, i.e., weakly correlated inputs result in more correlated responses and vice versa. Thus, we uncover a mechanism by which input correlation can be bidirectionally modulated, which is possible only because of high sharing of FSI inputs. We posit that the emergence of a correlation attractor leads to non-zero correlation level and variable rate trajectories of striatal responses across trials, hence beneficial for exploration in learning. However, given their role in across-trial variability, we argue that FSIs should be 'disengaged' from the MSNs during performance where stability across trials is required.