MapReduce-Based Improved Random Forest Model for Massive Educational Data Processing and Classification

Xu, Wei; Hoang, Vinh Truong

doi:10.1007/s11036-020-01699-w

Cited by 7 publications

(4 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, the idea of creating GBT to analyze and predict student performance was recently applied to the Brazilian public school system. The study proved that internal grades and number of absences are some of the major attributes that helped to generate good accuracy of the predicting model [45]. In addition, to forecast the dropout ratio of registered students for online and on-campus studies, the gradient boosted decision tree model has been applied several times and successfully proved its prediction capability [46][47][48].…”

Section: Gradient Boosted Treesmentioning

confidence: 88%

“…By building multiple trees, the approach minimizes the variance and typically provides more accurate results [44]. This method was applied several times on the educational dataset dealing with different problems [13,45,46]. Therefore, the separation of multiple attributes using splitting rules make it easy to understand and useful for prediction purposes.…”

Section: Random Forestmentioning

confidence: 99%

See 1 more Smart Citation

Intelligent Decision Support System for Predicting Student’s E-Learning Performance Using Ensemble Machine Learning

et al. 2021

View full text Add to dashboard Cite

Electronic learning management systems provide live environments for students and faculty members to connect with their institutional online portals and perform educational activities virtually. Although modern technologies proactively support these online sessions, students’ active participation remains a challenge that has been discussed in previous research. Additionally, one concern for both parents and teachers is how to accurately measure student performance using different attributes collected during online sessions. Therefore, the research idea undertaken in this study is to understand and predict the performance of the students based on features extracted from electronic learning management systems. The dataset chosen in this study belongs to one of the learning management systems providing a number of features predicting student’s performance. The integrated machine learning model proposed in this research can be useful to make proactive and intelligent decisions according to student performance evaluated through the electronic system’s data. The proposed model consists of five traditional machine learning algorithms, which are further enhanced by applying four ensemble techniques: bagging, boosting, stacking, and voting. The overall F1 scores of the single models are as follows: DT (0.675), RF (0.777), GBT (0.714), NB (0.654), and KNN (0.664). The model performance has shown remarkable improvement using ensemble approaches. The stacking model by combining all five classifiers has outperformed and recorded the highest F1 score (0.8195) among other ensemble methods. The integration of the ML models has improved the prediction ratio and performed better than all other ensemble approaches. The proposed model can be useful for predicting student performance and helping educators to make informed decisions by proactively notifying the students.

show abstract

Section: Gradient Boosted Treesmentioning

confidence: 88%

Section: Random Forestmentioning

confidence: 99%

Intelligent Decision Support System for Predicting Student’s E-Learning Performance Using Ensemble Machine Learning

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Mu et al [19] introduced the Pearson correlation coefficient to determine the optimal split attribute and split point during decision tree growth and trained decision trees in parallel using MapReduce technology. Xu W et al [20] calculated the information gain of various features on MapReduce computing framework by introducing a feature weighting system and improving existing data analysis with evaluation metrics. Chen et al [21] combined dataparallel and task-parallel optimization methods to reduce communication costs between data and workload imbalances.…”

Section: Introductionmentioning

confidence: 99%

A Fast Parallel Random Forest Algorithm Based on Spark

et al. 2023

View full text Add to dashboard Cite

To improve the computational efficiency and classification accuracy in the context of big data, an optimized parallel random forest algorithm is proposed based on the Spark computing framework. First, a new Gini coefficient is defined to reduce the impact of feature redundancy for higher classification accuracy. Next, to reduce the number of candidate split points and Gini coefficient calculations for continuous features, an approximate equal-frequency binning method is proposed to determine the optimal split points efficiently. Finally, based on Apache Spark computing framework, the forest sampling index (FSI) table is defined to speed up the parallel training process of decision trees and reduce data communication overhead. Experimental results show that the proposed algorithm improves the efficiency of constructing random forests while ensuring classification accuracy, and is superior to Spark-MLRF in terms of performance and scalability.

show abstract

“…Supervised classification is widely used in almost every area nowadays. Its application in medicine [1][2][3], agriculture [4][5][6], education [7][8][9][10], sports [11][12][13], and social behavior in business [14][15][16], among other areas [17][18][19], is indisputable. However, it is well known that there is no classifier having an overall superior performance to the remaining ones for all problems.…”

Section: Introductionmentioning

confidence: 99%

A Priori Determining the Performance of the Customized Naïve Associative Classifier for Business Data Classification Based on Data Complexity Measures

et al. 2022

View full text Add to dashboard Cite

In the supervised classification area, the algorithm selection problem (ASP) refers to determining the a priori performance of a given classifier in some specific problem, as well as the finding of which is the most suitable classifier for some tasks. Recently, this topic has attracted the attention of international research groups because a very promising vein of research has emerged: the application of some measures of data complexity in the pattern classification algorithms. This paper aims to analyze the response of the Customized Naïve Associative Classifier (CNAC) in data taken from the business area when some measures of data complexity are introduced. To perform this analysis, we used classification datasets from real-world related to business, 22 in total; then, we computed the value of nine measures of data complexity to compare the performance of the CNAC against other algorithms of the state of the art. A very important aspect of performing this task is the creation of an artificial dataset for meta-learning purposes, in which we considered the performance of CNAC, and then we trained a decision tree as meta learner. As shown, the CNAC classifier obtained the best results for 10 out of 22 datasets of the experimental study.

show abstract

MapReduce-Based Improved Random Forest Model for Massive Educational Data Processing and Classification

Cited by 7 publications

References 25 publications

Intelligent Decision Support System for Predicting Student’s E-Learning Performance Using Ensemble Machine Learning

Intelligent Decision Support System for Predicting Student’s E-Learning Performance Using Ensemble Machine Learning

A Fast Parallel Random Forest Algorithm Based on Spark

A Priori Determining the Performance of the Customized Naïve Associative Classifier for Business Data Classification Based on Data Complexity Measures

Contact Info

Product

Resources

About