Hidden hearing loss (HHL) is a deficit in auditory perception and speech intelligibility that occurs despite normal audiometric thresholds and results from noise exposure, aging, or myelin defects. While mechanisms causing perceptual deficits in HHL patients are still unknown, results from animal models indicate a role for peripheral auditory neuropathies in HHL. In humans, sound localization is particularly important for comprehending speech, especially in noisy environments, and its disruption may contribute to HHL. In this study, we hypothesized that neuropathies of cochlear spiral ganglion neurons (SGNs) that are observed in animal models of HHL disrupt the activity of neurons in the medial superior olive (MSO), a nucleus in the brainstem responsible for locating low-frequency sound in the horizontal plane using binaural temporal cues, leading to sound localization deficits. To test our hypothesis, we constructed a network model of the auditory processing system that simulates peripheral responses to sound stimuli and propagation of responses via SGNs to cochlear nuclei and MSO populations. To simulate peripheral auditory neuropathies, we used a previously developed biophysical SGN model with myelin defects at SGN heminodes (myelinopathy) and with loss of inner hair cell-SGN synapses (synaptopathy). Model results indicate that myelinopathy and synaptopathy in SGNs give rise to decreased interaural time difference (ITD) sensitivity of MSO cells, suggesting a possible mechanism for perceptual deficits in HHL patients. This model may be useful to understand downstream impacts of SGN-mediated disruptions on auditory processing and to eventually discover possible treatments for various mechanisms of HHL.