Facetwise Analysis of XCS for Problems With Class Imbalances

Abstract

Michigan-style learning classifier systems (LCSs) are online machine learning techniques that incrementally evolve distributed subsolutions which individually solve a portion of the problem space. As in many machine learning systems, extracting accurate models from problems with class imbalances-that is, problems in which one of the classes is poorly represented with respect to the other classes-has been identified as a key challenge to LCSs. Empirical studies have shown that Michigan-style LCSs fail to provide accurate subsolutions that represent the minority class in domains with moderate and large disproportion of examples per class; however, the causes of this failure have not been analyzed in detail. Therefore, the aim of this paper is to carefully examine the effect of class imbalances on different LCS components. The analysis focuses on XCS, which is the most-relevant Michigan-style LCS, although the models could be easily adapted to other LCSs. Design decomposition is used to identify five elements that are crucial to guaranteeing the success of LCSs in domains with class imbalances, and facetwise models that explain these different elements for XCS are developed. All theoretical models are validated with artificial problems. The integration of all these models enables us to identify the sweet spot where XCS is able to scalably and efficiently evolve accurate models of rare classes; furthermore, facetwise analysis is used as a tool for designing a set of configuration guidelines that have to be followed to ensure convergence. When properly configured, XCS is shown to be able to solve highly unbalanced problems that previously eluded solution

Addressing the Classification with Imbalanced Data: Open Problems and New Challenges on Class Distribution

“…This situation occurs when the concepts are represented within small clusters, which arise as a direct result of underrepresented subconcepts [11,40]. Although those small disjuncts are implicit in most of the problems, the existence of these small disjuncts highly increases the complexity of the problem in the case of imbalance because it becomes hard to know whether these examples represent an actual subconcept or are merely attributed to noise [41].…”

Learning local linear Jacobians for flexible and adaptive robot arm control

“…The number of movement iterations is set to 500,000-however, as mentioned above active learning does not occur at every iteration, but only when the approximation is inaccurate. The threshold h GA specifies how frequently the GA is activated and is increased to 200 in order to compensate for the imbalanced sampling as suggested in [19]. Towards the end of a run, condensation [32] is activated, that is, reproduction without mutation and crossover to remove evolutionary overhead.…”

Self-adaptation of parameters in a learning classifier system ensemble machine

“…Additionally, the optimal value of the learning rate depends on the generality and accuracy of the classifier being learned. The parameter β should be decreased for overgeneral rules, in which large fluctuations of the prediction p, prediction error and fitness f may occur (Butz et al, 2005;Orriols-Puig et al, 2009).…”

“…Nevertheless, some parameters have direct influence on classifier evaluation and cannot be simply self-adapted. For example, the learning rate controls updates of classifier parameters (among others, the fitness updates) and an incorrect value of β makes inaccurate classifiers over-fitted (Orriols-Puig et al, 2009). In (Hurst and Bull, 2003) it was shown that the adaptation of β at the individual level is "selfish" indeed and therefore the "enforced cooperation" method, which is dedicated to systems solving multi-step problems, was proposed.…”