Sound input critically influences the development and maintenance of neuronal circuits in the mammalian brain throughout life. We investigate the structural and functional plasticity of auditory neurons in response to various auditory experiences during development, adulthood, and aging. Using electrophysiology, computer simulation, and immunohistochemistry, we study the structural plasticity of the axon initial segment (AIS) in the medial nucleus of the trapezoid body (MNTB) from the auditory brainstem of the mice (either sex), in different ages and auditory environments. The structure and spatial location of the AIS of MNTB neurons depend on their functional topographic location along the tonotopic axis, aligning high- to low-frequency sound-responding neurons (HF or LF neurons). HF neurons dramatically undergo structural remodeling of the AIS throughout life. The AIS progressively shortens during development, is stabilized in adulthood, and becomes longer in aging. Sound inputs are critically associated with setting and maintaining AIS plasticity and tonotopy at various ages. Sound stimulation increases the excitability of auditory neurons. Computer simulation shows that modification of the AIS length, location, and diameter can affect firing properties of MNTB neurons in the developing brainstem. The adaptive capability of axonal structure in response to various auditory experiences at different ages suggests that sound input is important for the development and maintenance of the structural and functional properties of the auditory brain throughout life.