Background Non-adherence to anti-seizure medication (ASM) therapy is an important contributing factor to the higher mortality rate and treatment failure of epilepsy. This study aimed to determine the rate and factors associated with non-adherence to ASM therapy through the WHO five dimensions of medication adherence framework. Methods We conducted a cross-sectional study at an outpatient Neurology Clinic of a tertiary government hospital in Malaysia. Between March and July 2019, we identified 217 patients with a confirmed diagnosis of epilepsy, receiving oral ASM therapy and able to administer their medications. We performed a semi-structured interview to gather information on sociodemographic background, clinical and medication history, and perceptions on healthcare services. Adherence to ASM therapy was evaluated using the Medication Compliance Questionnaire (MCQ). Patient's illness perception was assessed by the Brief Illness Perception Questionnaire (B-IPQ). Results 208 patients participated in this study. The median age of the study participants was 35 years (IQR 26-44). 58.2% were females and majority, 55.8%, were from the Malay ethnic group. Based on the MCQ scoring, 89 patients (42.8%) were non-adherent. Multiple logistic regression demonstrated that being employed or students (adjusted odds ratio [aOR] 2.26, 95%CI: 1.19-4.29 p = 0.012) and having an average or below average perceived access to pharmacy services (aOR 2.94, 95%CI: 1.38-6.24, p = 0.005) were significant contributors to non-adherence.