Working memory (WM) provides flexible storage of information in service of upcoming behavioral goals. Some models propose specific fixed loci and mechanisms for the storage of visual information in WM, such as sustained spiking in parietal and prefrontal cortex during the maintenance of features. An alternative view is that information can be remembered in a flexible format that best suits current behavioral goals. For example, remembered visual information might be stored in sensory areas for easier comparison to future sensory inputs (i.e. a retrospective code) or might be remapped into a more abstract, output-oriented format and stored in motor areas (i.e. a prospective code). Here, we tested this hypothesis using a visual-spatial working memory task where the required behavioral response was either known or unknown during the memory delay period. Using fMRI and multivariate decoding, we found that there was less information about remembered spatial positions in early visual and parietal regions when the required response was known versus unknown. Further, a representation of the planned motor action emerged in primary somatosensory, primary motor, and premotor cortex on the same trials where spatial information was reduced in early visual cortex. These results suggest that the neural networks supporting WM can be strategically reconfigured depending on the specific behavioral requirements of canonical visual WM paradigms.