Previous research has consistently shown that high trait anxiety (HTA) reduces spatial and verbal working memory capacity. However, its effects on visual working memory (VWM) are inconsistent, and the underlying mechanisms remain unclear. This study investigates the impact of trait anxiety on VWM capacity and the role of filtering efficiency through two experiments. Experiment 1 utilized a change detection task (CDT) with 2–5 colored squares. The results indicated that low trait anxiety (LTA) individuals had significantly higher K scores than individuals with HTA, particularly under high memory load, indicating limitations in VWM capacity within the HTA group. Additionally, the contralateral delay activity (CDA) amplitude in the HTA group stabilized earlier than in the LTA group, further suggesting a deficit in VWM capacity among HTA individuals. Experiment 2 introduced neutral distractors. Under low load conditions, both groups were unaffected by the distractors. However, under high load, the HTA group was susceptible to interference, whereas the LTA group effectively filtered out distractors. CDA patterns revealed that individuals with HTA could filter distractors under low load but not under high load, whereas LTA individuals showed effective filtering under both conditions.