Combining diverse neural nets

Abstract

An appropriate use of neural computing techniques is to apply them to problems such as condition monitoring, fault diagnosis, control and sensing, where conventional solutions can be hard to obtain. However, when neural computing techniques are used, it is important that they are employed so as to maximise their performance, and improve their reliability. Their performance is typically assessed in terms of their ability to generalise to a previously unseen test set, although unless the training set is very carefully chosen, 100% accuracy is rarely achieved. Improved performance can result when sets of neural nets are combined in ensembles and ensembles can be viewed as an example of the reliability through redundancy approach that is recommended for conventional software and hardware in safety-critical or safety-related applications. Although there has been recent interest in the use of neural net ensembles, such techniques have yet to be applied to the tasks of condition monitoring and fault diagnosis. In this paper, we focus on the benefits of techniques which promote diversity amongst the members of an ensemble, such that there is a minimum number of coincident failures. The concept of ensemble diversity is considered in some detail, and a hierarchy of four levels of diversity is presented. This hierarchy is then used in the description of the application of ensemble-based techniques to the case study of fault diagnosis of a diesel engine.

Untitled

“…Ghosh (1996b, 1999) do not mention the case of negative correlation although it clearly supports their thesis that the smaller the correlation, the better the ensemble. A negative correlation between the continuousvalued outputs has been sought, predominantly by altering the available training set or parameters of the classifier (Dietterich, 2000a;Hashem, 1999;Krogh & Vedelsby, 1995;Liu & Yao, 1999;Opitz & Shavlik, 1999;Parmanto et al, 1996;Giacinto & Roli, 2001;Rosen, 1996;Sharkey & Sharkey, 1997;Skalak, 1996;Tumer & Ghosh, 1999). -When classifiers output class labels, the classification error can be decomposed into bias and variance terms (also called 'spread') (Bauer & Kohavi, 1999;Breiman, 1999;Kohavi & Wolpert, 1996) or into bias and spread terms.…”

Testing Ensembles for Intrusion Detection: On the Identification of Mutated Network Scans

“…One of the most widespread and useful techniques in order to avoid such problems is the ensemble learning scheme [19], [20]. The main idea behind this kind of meta-algorithms is to train several slightly different simpler classifiers and combine their results in order to improve the results obtained by a single, usually more complex, one [21].…”

Clustering Ensemble for Spam Filtering

“…The underlying reason for increased reliability through the use of ensembles is that di erent classi cation algorithms will show di erent patterns of generalization. More formal explanations of the way ensembles can improve performance may be found in [5,6].…”