Smoothed Emphasis for Boosting Ensembles

Boosting ensembles with controlled emphasis intensity

“…For the sake of clarity, let us insist: There are three components in (1) . The first is the constant term α: When it takes high values, the intensity of emphasis is reduced, and this can be beneficial when solving some problems.…”

“…With respect to the auxiliary machine, or guide, which provides the values of o ( n) to be used in (1), there are evidences of the ad-vantage of using relatively powerful machines offering outputs not very different from those expected with the emphasized design. Thus, using the partial ensemble which is available when training each learner is an appropriate selection: This partial ensemble will be good enough in the final steps of the building process, and the similarity is obvious.…”

“…Of course, many other error and proximity measures could be employed in (1), and results would be better or worse depending on the database under analysis. But we invoke [5] to defend that our objective is to check if moderating the emphasis with α = 0 can be beneficial, and not to explore how different measures work in different problems.…”

“…A second step is taken in [1,2], where the above mixed empha-sis is also applied to the K nearest neighbors of each sample, and the overall weight for each sample is a convex combination of the individual and the average neighbor emphasis. A version in which the combination of the error and the proximity terms is selected for each learner according to the minimization of the edge parameter, which is called DWK-RA (Dynamic Weighting K-neighbour Real Adaboost), provides excellent experimental results.…”