The objective of this study was to identify clinical parameters predicting either a pathological EEG or a subsequent epileptic seizure (SES), based on the relation between paroxysmal EEG abnormalities and clinical features in children who presented at least one febrile seizure (FS). We collected data of children who presented to our department during the period 2013 to 2018 for EEG recording as part of their febrile seizure assessment. Only children aged between 1 month to 5 years were included. Both the clinical and EEG data were retrospectively collected and statistically studied. We performed a detailed analysis of the EEG recordings. SES was identified for patients with sufficient follow-up. A total of 120 children were included in the study, of whom 48% had EEG abnormalities. Psychomotor retardation (p = 0.002), completion of an EEG within 7 days of the last FS (p = 0.046), and late age (> 3 years) of the first FS onset (p = 0.021) were significantly associated with a pathological EEG. In multivariate analysis, performing early EEG (< 7 days from the last FS) (odds ratio [OR]: 2.35; p = 0.043; confidence interval [CI]: 1.028–5.375) and psychomotor retardation (OR: 4.19; p = 0.008; CI: 1.46–12) were independent predictors of a pathological EEG. Of 120 patients, 45 had a follow-up. However, only 10 (22.22%) had SES. Children with SES tended more to have a psychomotor delay, compared with children without SES (50% vs. 14.28%, p = 0.029). Moreover, the percentage of initial abnormal EEG in patients with SES was significantly higher than those without SES (70% vs. 34.28%, p = 0.05). Even though some FS characteristics predict EEG abnormalities, they are not always associated with SES. We highlight the importance of performing an EEG in the group of children who had both FS and psychomotor retardation. This is most likely the group at the highest risk of developing epilepsy.