Selecting relevant information while ignoring irrelevant distractors is critical for every cognitive function. Visual working memory (VWM) load is 1 factor that can modulate attentional selection; however, previous studies yielded inconsistent results concerning whether VWM load increases or decreases attentional selection. In the current study, we manipulated VWM load while controlling the size of attentional zoom, which could have influenced distractor interference in prior studies. We found that increasing the size of attentional zoom resulted in greater distractor interference; however, increasing VWM load did not modulate attentional selection when attentional zoom was held narrow. The current findings suggest that the interaction between selective attention and VWM can be better understood by dissociating the size of attentional focus and VWM load.