Confusion emotion in a learning environment can motivate the learner, but prolonged confusion hinders the learning process. Recognizing confused learners is possible; nevertheless, finding them requires a lot of time and effort. Due to certain restrictions imposed by the settings of an online learning environment, the recognition of confused students is a big challenge for educators. Therefore, novel technologies are necessary to handle such crucial difficulties. Lately, Electroencephalography (EEG)-based emotion recognition systems have been rising in popularity in the domain of Education Technology. Such systems have been utilized to recognize the confusion emotion of learners. Numerous studies have been conducted to recognize confusion emotion through this system since 2013, and because of this, a systematic review of the methodologies, feature sets, and utilized classifiers is a timely necessity. This article presents the findings of the review conducted to achieve this requirement. We summarized the published literature in terms of the utilized datasets, feature preprocessing, feature types for model training, and deployed classifiers in terms of shallow machine learning and deep learning-based algorithms. Moreover, the article presents a comparison of the prediction accuracies of the classifiers and illustrates the existing research gaps in confusion emotion recognition systems. Future study directions for potential research are also suggested to overcome existing gaps.