Antiepileptic drugs impair episodic memory in patients with epilepsy, but this effect has so far only been examined with tests that do not provide first-person experience-an aspect that is crucial for episodic memory. Virtual reality techniques facilitate the development of ecologically valid tests. In the present study, we measure the effect of antiepileptic drug changes in a within-subject design using a virtual reality test in order to provide direct evidence for effects of antiepileptic drugs on episodic memory. Among 106 recruited patients, 97 participated in a virtual reality test up to six times during a 4-day hospitalization, and 78 patients underwent changes in drug load during this period. There were six parallel versions of a virtual town test, with immediate recall and delayed recall after about 12 h. The test requires recall of elements, details, sequence of experience, and egocentric and allocentric spatial memory. We determined drug load by defined daily dose, and compared test performance at lowest antiepileptic drug load to highest antiepileptic drug load. Across the six towns, performance was lower in delayed compared to immediate recall. There was an overall effect of medication when comparing patients taking vs. not taking antiepileptic drugs and/or psychoactive drugs (p = 0.005). Furthermore, there was a within-subject effect of antiepileptic drug load (p = 0.01), indicating lower test performance at higher drug load. There was no effect of gender, daytime, circadian type, depression, seizures, lesions, and epilepsy. For patients with temporal lobe epilepsy, there was no effect of lateralization. The present study provides direct evidence for episodic memory impairment due to antiepileptic drugs, suggesting that a small change in drug load can matter. This study can serve as a proof of principle for the methodology, but a larger sample is needed to examine the differential effects of individual antiepileptic drugs.