Engineering Multiversion Neural-Net Systems

Abstract

In this paper we address the problem of constructing reliable neural-net implementations, given the assumption that any particular implementation will not be totally correct. The approach taken in this paper is to organize the inevitable errors so as to minimize their impact in the context of a multiversion system, i.e., the system functionality is reproduced in multiple versions, which together will constitute the neural-net system. The unique characteristics of neural computing are exploited in order to engineer reliable systems in the form of diverse, multiversion systems that are used together with a "decision strategy" (such as majority vote). Theoretical notions of "methodological diversity" contributing to the improvement of system performance are implemented and tested. An important aspect of the engineering of an optimal system is to overproduce the components and then choose an optimal subset. Three general techniques for choosing final system components are implemented and evaluated. Several different approaches to the effective engineering of complex multiversion systems designs are realized and evaluated to determine overall reliability as well as reliability of the overall system in comparison to the lesser reliability of component substructures.

Ensembles of Learning Machines

“…From this point of view heuristic rules, as the "choose the best" or the "choose the best in the class", using classifiers of different types strongly reduce the computational complexity of the selected phase, as the evaluation of different classifier subsets is not required [103]. Moreover test and select methods implicitly include a "production stage", by which a set of classifiers must be generated.…”

A Study on the Use of Multiobjective Genetic Algorithms for Classifier Selection in FURIA-based Fuzzy Multiclassifiers

“…In general, the selection of a subset of classifiers is done using the OCS strategy [13,30], in which a large set of classifiers is produced and then selected to extract the best performing subset. GAs are a popular technique within this strategy.…”

“…As a consequence, fuzzy rulebased multiclassification systems (FRBMCSs) were incorporated into an overproduce-and-choose strategy (OCS) [13]. This MCS desing algorithm is based on the generation of a large number of component classifiers, and a subsequent selection of the subset of them best cooperating.…”

Eigenclassifiers for combining correlated classifiers

“…BAG: The original Bagging algorithm proposed in [7]. We train decision tree ensembles of size 5,10,15,20,25,30 and choose the one with the best val-B accuracy.…”

“…That is, we are interested in both pruning the inaccurate ones and also to keep a check on complexity, we want to prune the redundant. ''Diversity'' measures have been proposed [23,22] and one possibility is to have an incremental, forward search where we add a classifier to an ensemble if it is diverse or adds to accuracy [9,11,35,49,42], or another possibility is to have a decremental, backward search where a classifier is removed or pruned if it is not diverse enough or if its removal does not increase error [30,27]. In this work, we propose an alternative method which combines base classifiers using principal component analysis (PCA) to get uncorrelated eigenclassifiers.…”