The anterior temporal lobe (ATL) makes a critical contribution to semantic cognition. However, the functional connectivity of the ATL and the functional network underlying semantic cognition has not been elucidated. In addition, subregions of the ATL have distinct functional properties and thus the potential differential connectivity between these subregions requires investigation. We explored these aims using both resting-state and active semantic task data in humans in combination with a dual-echo gradient echo planar imaging (EPI) paradigm designed to ensure signal throughout the ATL. In the resting-state analysis, the ventral ATL (vATL) and anterior middle temporal gyrus (MTG) were shown to connect to areas responsible for multimodal semantic cognition, including bilateral ATL, inferior frontal gyrus, medial prefrontal cortex, angular gyrus, posterior MTG, and medial temporal lobes. In contrast, the anterior superior temporal gyrus (STG)/superior temporal sulcus was connected to a distinct set of auditory and language-related areas, including bilateral STG, precentral and postcentral gyri, supplementary motor area, supramarginal gyrus, posterior temporal cortex, and inferior and middle frontal gyri. Complementary analyses of functional connectivity during an active semantic task were performed using a psychophysiological interaction (PPI) analysis. The PPI analysis highlighted the same semantic regions suggesting a core semantic network active during rest and task states. This supports the necessity for semantic cognition in internal processes occurring during rest. The PPI analysis showed additional connectivity of the vATL to regions of occipital and frontal cortex. These areas strongly overlap with regions found to be sensitive to executively demanding, controlled semantic processing.