Cell cycle associated protein 1 (Caprin1) is an RNA-binding protein that can regulate the cellular post-transcriptional response to stress. It is a component of both stress granules and neuronal RNA granules and is implicated in neurodegenerative disease, synaptic plasticity and long-term memory formation. Our previous work suggested that Caprin1 also plays a role in the response of the cochlea to stress. Here, targeted inner ear-deletion of Caprin1 in mice leads to an early onset, progressive hearing loss. Auditory brainstem responses from Caprin1-deficient mice show reduced thresholds, with a significant reduction in wave-I amplitudes compared to wildtype. Whilst hair cell structure and numbers were normal, the inner hair cell-spiral ganglion neuron (IHC-SGN) synapse revealed abnormally large post-synaptic GluA2 receptor puncta, a defect consistent with the observed wave-I reduction. Unlike wildtype mice, mild-noise induced hearing threshold shifts in Caprin1-deficient mice did not recover. Oxidative stress triggered TIA-1/HuR positive stress granule formation in ex-vivo cochlear explants from Caprin1-deficient mice, showing that stress granules could still be induced. Taken together, these findings suggest that Caprin1 plays a key role in maintenance of auditory function, where it regulates the normal status of the IHC-SGN synapse.