The existence of a neural representation for whole words(i.e.,a lexicon) is a common feature of many models of speech processing. Prior studies have provided evidence for a visual lexicon containing representations of whole written words in an area of the ventral visual stream known as the "Visual Word Form Area" (VWFA). Similar experimental support for an auditory lexicon containing representations of spoken words has yet to be shown. Using fMRI rapid adaptation techniques, we provide evidence for an auditory lexicon in the "Auditory Word Form Area" (AWFA) in the human left anterior superior temporal gyrus that contains representations highly selective for individual spoken words. Furthermore, we show that familiarization with novel auditory words sharpens their selectivity in the AWFA. These findings reveal strong parallels in how the brain represents written and spoken words, showing convergent processing strategies across speech modalities in the visual and auditory ventral streams.