Data Mining Solution for Assessing Brazilian Secondary School Quality Based on ENEM and Census Data

Abstract

This paper presents a data mining solution for assessing the quality of Brazilian private secondary schools based on the official school survey and students tests. Following the CRISP-DM method, after the problem interpretation and modeling, these two data sources yearly collected have been transformed to the school granularity level embedding data and expert´s knowledge and have been integrated in a single data set with the national school code as primary key. Further transformations on the joint data set embedded additional knowledge and made the format compatible with the artificial intelligence techniques applied for knowledge extraction. Logistic regression was applied for producing a propensity score for good schools, decision tree applied for extracting the sequential decision making a human would follow and rules were induced for supporting the explanation of a decision based on the score. The AUC_ROC and Max_KS were used for assessing the propensity score performance and, coverage, confidence and lift were used for assessing the quality of the rules induced by the A Priori algorithm, together with the human knowledge available on the literature. The results showed that this domaindriven data mining approach was successful in modeling the problem and validating educational public policies.

Beyond scores: A machine learning approach to comparing educational system effectiveness

“…By modeling learning, they seek to predict student achievement based on variables collected in the LSA questionnaires [ 16 ]. Regression models are the most used approach, although classification has also been used, and statistical separatrices [ 15 , 17 ], absolute thresholds [ 18 ] or unsupervised learning utilizing clusters has been used to perform the class labels [ 19 ]. As detected in previous studies, tree-based algorithms have been the most used technique for both regression and classification tasks [ 15 ].…”

Analysis of the Impact of the Pandemic on Social Inequalities in Enem 2019 and 2020 using Machine Learning

“…Researchers Adeodato (2016) and Adeodato and Silva (2020) justified that students in the upper quartile of the average good perform well. In this work, this methodology was followed, in which the student's performance is the simple average of the five tests that compromise the exam.…”

Leveraging Causal Reasoning in Educational Data Mining: An Analysis of Brazilian Secondary Education

“…The findings reporting is often featured by ranking feature importance [3,26]. Additionally, some studies probed their results by exploring additional explainable techniques such as partial dependence plots [2,[27][28][29], decision trees, and rules [30,31].…”