It is a fundamental question whether the medial temporal lobe (MTL) supports only long-term memory encoding, or contributes to working memory (WM) processes as well. Recent data suggest that the MTL is activated whenever multiple items or item features are being maintained in WM. This may rely on interactions between the MTL or the prefrontal cortex (PFC) and content-specific areas in the inferior temporal (IT) cortex. Here, we investigated the neural mechanism through which the MTL, PFC, and IT cortex interact during WM maintenance. First, we quantified phase synchronization of intracranial EEG data in epilepsy patients with electrodes in both regions. Second, we used directional coupling analysis to study whether oscillatory activity in the IT cortex drives the MTL or vice versa. Finally, we investigated functional connectivity in functional magnetic resonance imaging data of healthy subjects with seeds in the MTL and PFC. With increasing load, EEG phase synchronization between the IT cortex and anterior parahippocampal gyrus and within the MTL increased. Coupling was bidirectional in all load conditions, but changed toward an increased top-down (anterior parahippocampal gyrus 3 IT) coupling in the high gamma range (51-75 Hz) with increasing load. Functional connectivity between the MTL seed and the visual association cortex increased with load, but activity within the MTL and the PFC correlated with fewer voxels, suggesting that more specific neural networks were engaged. These data indicate that WM for multiple items depends on an increased strength of top-down control of activity within the IT cortex by the MTL.