The triad of noise-generated, drug-induced, and age-related hearing loss is the major cause of acquired sensorineural hearing loss (ASNHL) in modern society. Although these three forms of hearing loss display similar underlying mechanisms, detailed studies have revealed the presence of sex differences in the auditory system both in human and animal models of ASNHL. However, the sexual dimorphism of hearing varies among noise-induced hearing loss (NIHL), ototoxicity, and age-related hearing loss (ARHL). Importantly, estrogen may play an essential role in modulating the pathophysiological mechanisms in the cochlea and several reports have shown that the effects of hormone replacement therapy on hearing loss are complex. This review will summarize the clinical features of sex differences in ASNHL, compare the animal investigations of cochlear sexual dimorphism in response to the three insults, and address how estrogen affects the auditory organ at molecular levels.