We assessed the impact of age at onset of epilepsy and duration and frequency of seizures on cognitive development in children less than 3 years old. Retrospective analysis was conducted on clinical data and neuropsychological testing of 33 infants with epilepsy. Developmental quotients were calculated and were correlated with age at epilepsy onset, duration of epilepsy, seizure frequency, brain pathology, and types of seizures (with/without spasms) as potential predictors. Infants with longer duration and earlier onset of epilepsy performed worse on developmental neuropsychological testing. Regression analyses showed that age at epilepsy onset and percentage of life with epilepsy were both strongly associated (regression model P < 0.0001) with developmental quotient. There was no correlation with seizure frequency. Infants with spasms had worse developmental quotients than infants without spasms (P < 0.001). These results suggest that duration of epilepsy and age at onset may be the best developmental predictors during the first years of life in patients with epilepsy. Early aggressive intervention should be considered.