Pareto Cooperative-Competitive Genetic Programming: A Classification Benchmarking Study

In this paper we investigate using multi-objective genetic programming to evolve a feature extraction stage for multiple-class classifiers. We find mappings which transform the input space into a new, multi-dimensional decision space to increase the discrimination between all classes; the number of dimensions of this decision space is optimized as part of the evolutionary process. A simple and fast multi-class classifier is then implemented in this multi-dimensional decision space. Mapping to a single decision space has significant computational advantages compared to k-class-to-2-class decompositions; a key design requirement in this work has been the ability to incorporate changing priors and/or costs associated with mislabeling without retraining. We have employed multi-objective optimization in a Pareto framework incorporating solution complexity as an independent objective to be minimized in addition to the main objective of the misclassification error. We thus give preference to simpler solutions which tend to generalize well on unseen data, in accordance with Occam's Razor. We obtain classification results on a series of benchmark problems which are essentially identical to previous, more complex decomposition approaches. Our solutions are much simpler and computationally attractive as well as able to readily incorporate changing priors/costs. In addition, we have also applied our approach to the KDD-99 intrusion detection dataset and obtained results which are highly competitive with the KDD-99 Cup winner but with a significantly simpler classification framework.

Section: Handling Multi-class Datasetsmentioning

confidence: 99%

“…Tables 4,5,6,7,8 show the confusion matrices for most of the UCI datasets in Table 3, for the individuals with the best test error of the ten training repetitions. (We omit the confusion matrices for the WIN, TGD and IRIS datasets as the best performing individuals all achieved 100% accuracy.)…”

Section: Uci and Statlog Datasetsmentioning

confidence: 99%

Multi-class pattern classification using single, multi-dimensional feature-space feature extraction evolved by multi-objective genetic programming and its application to network intrusion detection

Badran

Rockett

2011

“…One potential drawback of the approach is that the exemplar archives might grow considerably. However, enforcing a finite archive through the introduction of diversity measures (such as fitness sharing) has been empirically shown to be effective for efficiently decoupling fitness evaluation from training partition cardinality under batch (offline) learning [132,133]. The extension to sliding window interfaces and single pass constraints for online learning indicate that it is possible to match the performance of multi-pass batch algorithms when there is no labelling error [8].…”

Section: Samplingmentioning

confidence: 99%

Evolutionary model building under streaming data for classification tasks: opportunities and challenges

Heywood

2014

Self Cite

Streaming data analysis potentially represents a significant shift in emphasis from schemes historically pursued for offline (batch) approaches to the classification task. In particular, a streaming data application implies that: (1) the data itself has no formal 'start' or 'end'; (2) the properties of the process generating the data are non-stationary, thus models that function correctly for some part(s) of a stream may be ineffective elsewhere; (3) constraints on the time to produce a response, potentially implying an anytime operational requirement; and (4) given the prohibitive cost of employing an oracle to label a stream, a finite labelling budget is necessary. The scope of this article is to provide a survey of developments for model building under streaming environments from the perspective of both evolutionary and non-evolutionary frameworks. In doing so, we bring attention to the challenges and opportunities that developing solutions to streaming data classification tasks are likely to face using evolutionary approaches.

“…In effect team members were learning to act everywhere, potentially negating insight into solutions. Recent approaches for addressing such a drawback represent the team as an archive of individuals independent from the population [31,32] or apply some form of ensemble selection post training [5,13]. Multi-population Teams are composed by sampling individuals from n different populations.…”

Section: Evolutionary Problem Decompositionmentioning

confidence: 99%

“…Naturally, maintaining sufficient diversity within the population becomes very important. One approach to this is to place a lot of emphasis on designing fitness functions that explicitly capture diversity as well as overall accuracy (of the team) e.g., negative correlation [5,13,28], strongly typed GP [21], local membership functions [31][32][33]; frequently under the context of multi-objective fitness formulations. One potential disadvantage of the single population model is that a recently introduced offspring can disrupt good quality team behaviour, as recognized in the 'profit sharing problem' of learning classifier systems [42].…”

Section: Evolutionary Problem Decompositionmentioning

confidence: 99%

Symbiotic coevolutionary genetic programming: a benchmarking study under large attribute spaces

Doucette

McIntyre

Lichodzijewski

et al. 2011

Self Cite

Classification under large attribute spaces represents a dual learning problem in which attribute subspaces need to be identified at the same time as the classifier design is established. Embedded as opposed to filter or wrapper methodologies address both tasks simultaneously. The motivation for this work stems from the observation that team based approaches to Genetic Programming (GP) have the potential to design multiple classifiers per class-each with a potentially unique attribute subspace-without recourse to filter or wrapper style preprocessing steps. Specifically, competitive coevolution provides the basis for scaling the algorithm to data sets with large instance counts; whereas cooperative coevolution provides a framework for problem decomposition under a bid-based model for establishing program context. Symbiosis is used to separate the tasks of team/ ensemble composition from the design of specific team members. Team composition is specified in terms of a combinatorial search performed by a Genetic Algorithm (GA); whereas the properties of individual team members and therefore subspace identification is established under an independent GP population. Teaming implies that the members of the resulting ensemble of classifiers should have explicitly non-overlapping behaviour. Performance evaluation is conducted over data sets taken from the UCI repository with 649-102,660 attributes and 2-10 classes. The resulting teams identify attribute spaces 1-4 orders of magnitude smaller than under the original data set. Moreover, team members generally consist of less than 10 instructions; thus, small attribute subspaces are not being traded for opaque models.