It is generally appreciated that storing memories of specific events in the mammalian brain, and associating features of the environment with behavioral outcomes requires fine-tuning of the strengths of connections between neurons through synaptic plasticity. It is less understood whether the organization of neuronal circuits comprised of multiple distinct neuronal cell types provides an architectural prior that facilitates learning and memory by generating unique patterns of neuronal activity in response to different stimuli in the environment, even before plasticity and learning occur. Here we simulated a neuronal network responding to sensory stimuli, and systematically determined the effects of specific neuronal cell types and connections on three key metrics of neuronal sensory representations: sparsity, selectivity, and discriminability. We found that when the total amount of input varied considerably across stimuli, standard feedforward and feedback inhibitory circuit motifs failed to discriminate all stimuli without sacrificing sparsity or selectivity. Interestingly, networks that included dedicated excitatory feedback interneurons based on the mossy cells of the hippocampal dentate gyrus exhibited improved pattern separation, a result that depended on the indirect recruitment of feedback inhibition. These results elucidate the roles of cellular diversity and neural circuit architecture on generating neuronal representations with properties advantageous for memory storage and recall.