Improving boosting methods with a stable loss function handling outliers

Abstract

In classification problems, the occurrence of abnormal observations is often encountered. How to obtain a stable model to deal with outliers has always been a subject of widespread concern. In this article, we draw on the ideas of the AdaBoosting algorithm and propose a asymptotically linear loss function, which makes the output function more stable for contaminated samples, and two boosting algorithms were designed, based on two different way of updating, to handle outliers. In addition, a skill for overcoming the instability of Newton's method when dealing with weak convexity is introduced. Several samples, where outliers were artificially added, show that the Discrete L-AdaBoost and Real L-AdaBoost Algorithms find the boundary of each category consistently under the condition where data is contaminated. In real world data sets, we show the effectiveness of suggested algorithms by comparison with other two ensemble learning methods, especially for large dataset.

A two-stage optimized robust kernel density estimation for Bayesian classification with outliers

“…These samples have a higher chance of being learned by the subsequent classifiers, and they are forced to optimize the margin distribution on these samples. However, when the noise-detection assumption does not hold, the algorithm often performs unsatisfactorily [14]. Boosting strategy requires training multiple classifiers in a serial manner, which can be a time-consuming procedure.…”