Impact of an Extra Layer on the Stacking Algorithm for Classification Problems

Abstract

Classifying and making decisions are tasks performed by any human being in their daily lives. Learning algorithms have been widely studied as tools to aid information management, with an objective to maximize the generalization capacity. Learning algorithms can be used individually or as a committee of machines (ensembles). An ensemble uses the solutions provided by several machines, making different combinations with them to reach a final decision, such as multi-layer algorithm stacking. When combining combination methods, one arrives at a three-layered architecture, which is the focus of this article. The objective of this work was to evaluate the influence of adding one more layer in the stacking metalearning algorithm in other to obtain accuracy, area under ROC and time in relation to the lower layers, under the influence of the experiment, database and level factors. It was possible to conclude that, statistically, classifiers of the extra layer presented, in a general way, better performance in terms of accuracy and area. However, time grew sharply at each top layer added.

Machine Learning in Python: Main Developments and Technology Trends in Data Science, Machine Learning, and Artificial Intelligence

“…Stacking algorithms, one of the more common methods for combining models, train an aggregator model on the predictions of a set of individual models so that it learns how to combine the individual predictions into one final prediction [41]. Common stacking variants also include meta features [42] or implement multiple layers of stacking [43], which is also known as multi-level stacking. Scikit-learn compatible stacking classifiers and regressors have been available in Mlxtend since 2016 [44] and were also recently added to Scikit-learn in v0.22.…”

“…Nonetheless, several PyTorch-based projects emerged over the years that aid the process of implementing neural networks for different use-cases, making code more compact while simplifying the model training. Notable examples of such libraries are Skorch 39 , which provides a Scikit-learn compatible API on top of PyTorch, Ignite 40 , Torchbearer 41 [121], Catalyst 42 , and PyTorch Lightning 43 .…”

Multilayered Ensemble Learning for short-term forecasting in agro-climatology

“…RMSE Comparison between the 4 layer's algorithms for temperature forecast on Station_1.We base our experimental study on the one from[6] about the impact of an Extra Layer on the Stacking Algorithm for classification which affirms that an extra layer improves classification task. The table 1 present the RMSE (Root mean Square Error) obtained for each algorithm of each layer of our prediction method.…”

Systematic Review on AI-Blockchain Based E-Healthcare Records Management Systems

“…This article brings machine learning and data mining together for a joint discussion because both disciplines are based on data science and frequently cross [44]. However, there are a few fundamental differences between data mining and machine learning.…”