Deficits in word retrieval are a hallmark of a variety of neurological illnesses spanning from dementia to traumatic injuries. The role of the dominant temporal lobe in fluent naming has been characterized by lesional analyses, functional imaging, and intracranial recordings, but limitations of each of these measures preclude a clear assessment of which specific constituent of the temporal lobe is critical for naming. We studied a large cohort of patients undergoing surgical resections or laser ablations of the dominant temporal lobe for medically intractable epilepsy (n=95). These techniques are exceedingly effective for seizure control but often result in language declines, particularly in confrontation naming, which can be socio-economically disabling. We used a multivariate voxel-based lesion symptom mapping analysis to localize brain regions significantly associated with visual object naming deficits. We observed that posterior inferior temporal regions, centered around the middle fusiform gyrus, were significantly associated with a decline in confrontation naming. Furthermore, we found that the posterior margin of anterior temporal lobectomies was linearly correlated to a decline in visual naming with a clinically significant decline occurring once the resection extended 6 cm from the anterior tip of the temporal lobe. We integrated these findings with electrocorticography during naming in a subset of this population and found that the majority of cortical regions whose resection was associated with a significant decline overlapped with regions that were functionally most active prior to articulation. Importantly, these loci coincide with the sites of susceptibility artifacts during echo-planar imaging, which explains why this region has not previously been implicated. Taken together, these data highlight the crucial contribution of the posterior ventral temporal cortex in lexical access and its important role in the pathophysiology of anomia following temporal lobe resections. Surgical strategies, including the use of laser ablation to target the medial temporal lobe as well as microsurgical approaches, should attempt to preserve this region to mitigate postoperative language deficits.