Democracy in neural nets: Voting schemes for classification

Battiti, Roberto; Colla, Anna Maria

doi:10.1016/0893-6080(94)90046-9

Cited by 273 publications

(118 citation statements)

References 9 publications

Supporting

Mentioning

111

Contrasting

Unclassified

Order By: Relevance

“…Note that the combination of different models for improving the overall classi cation performance has been extensively studied in the neural computation literature; see, for instance, Battiti and Colla (1994) and Husmeier (1999). We here borrow a term frequently used in neural networks research and refer to (29) as a committee of HMMs.…”

Section: A Committee Of Hmmsmentioning

confidence: 99%

Detection of Recombination in DNA Multiple Alignments with Hidden Markov Models

Husmeier

Wright²

2001

Journal of Computational Biology

View full text Add to dashboard Cite

Conventional phylogenetic tree estimation methods assume that all sites in a DNA multiple alignment have the same evolutionary history. This assumption is violated in data sets from certain bacteria and viruses due to recombination, a process that leads to the creation of mosaic sequences from different strains and, if undetected, causes systematic errors in phylogenetic tree estimation. In the current work, a hidden Markov model (HMM) is employed to detect recombination events in multiple alignments of DNA sequences. The emission probabilities in a given state are determined by the branching order (topology) and the branch lengths of the respective phylogenetic tree, while the transition probabilities depend on the global recombination probability. The present study improves on an earlier heuristic parameter optimization scheme and shows how the branch lengths and the recombination probability can be optimized in a maximum likelihood sense by applying the expectation maximization (EM) algorithm. The novel algorithm is tested on a synthetic benchmark problem and is found to clearly outperform the earlier heuristic approach. The paper concludes with an application of this scheme to a DNA sequence alignment of the argF gene from four Neisseria strains, where a likely recombination event is clearly detected.

show abstract

Section: A Committee Of Hmmsmentioning

confidence: 99%

Detection of Recombination in DNA Multiple Alignments with Hidden Markov Models

Husmeier

Wright²

2001

Journal of Computational Biology

View full text Add to dashboard Cite

show abstract

“…Although it is still possible that the same or very similar rules will be generated between different levels and classes even when using the method described in this section, the probability (dependant on the make-up of the training set) of this happening is smaller, when including the examples belonging to sibling classes as negative examples when inducing rules. Recall that to make an effective ensemble it is very important that the component classifiers be diverse, even if at the expense of some accuracy [34], [5], [1].…”

Section: Technical Details Of the Hehrs Methodsmentioning

confidence: 99%

“…Skalak [34] discusses an example of this phenomenon where a classifier that is 69% accurate is combined with classifiers that are 23% accurate and 25% accurate, and this boosts overall accuracy to 88%. The diversity of the component classifiers is very important in ensemble approaches, because component classifiers must make different errors to make the overall ensemble more accurate [5], [1]. There is often a trade-off between accuracy and diversity in classifiers, as it is often easier to make more diverse (uncorrelated) classifiers when the classification accuracies of the individual classifiers are lowered.…”

Section: Hierarchical Ensemble Of Hierarchical Rule Sets (Hehrs)mentioning

confidence: 99%

Hierarchical classification of protein function with ensembles of rules and particle swarm optimisation

Holden

Freitas

2008

Soft Comput

View full text Add to dashboard Cite

This paper focuses on hierarchical classification problems where the classes to be predicted are organized in the form of a tree. The standard top-down divide and conquer approach for hierarchical classification consists of building a hierarchy of classifiers where a classifier is built for each internal (non-leaf) node in the class tree. Each classifier discriminates only between its child classes. After the tree of classifiers is built, the system uses them to classify test examples one class level at a time, so that when the example is assigned a class at a given level, only the child classes need to be considered at the next level. This approach has the drawback that, if a test example is misclassified at a certain class level, it will be misclassified at deeper levels too. In this paper we propose hierarchical classification methods to mitigate this drawback. More precisely, we propose a method called Hierarchical Ensemble of Hierarchical Rule Sets (HEHRS), where different ensembles are built at different levels in the class tree and each ensemble consists of different rule sets built from training examples at different levels of the class tree. We also use a Particle Swarm Optimisation (PSO) algorithm to optimise the rule weights used by HEHRS to combine the predictions of different rules into a class to be assigned to a given test example. In addition, we propose a variant of a method to mitigate the aforementioned drawback of top-down classification. These three types of methods are compared against the standard top-down hierarchical classification method in six challenging bioinformatics datasets, involving the prediction of protein function. Overall HEHRS with the rule weights optimised by the PSO algorithm obtains the best predictive accuracy out of the four types of hierarchical classification method.

show abstract

“…In this last decade one of the main research areas in machine learning has been represented by methods for constructing ensembles of learning machines. Although in the literature [86,129,130,69,61,23,33,12,7,37] a plethora of terms, such as committee, classifier fusion, combination, aggregation and others are used to indicate sets of learning machines that work together to solve a machine learning problem, in this paper we shall use the term ensemble in its widest meaning, in order to include the whole range of combining methods. This variety of terms and specifications reflects the absence of an unified theory on ensemble methods and the youngness of this research area.…”

Section: Introductionmentioning

confidence: 99%

Ensembles of Learning Machines

2002

View full text Add to dashboard Cite

Abstract. Ensembles of learning machines constitute one of the main current directions in machine learning research, and have been applied to a wide range of real problems. Despite of the absence of an unified theory on ensembles, there are many theoretical reasons for combining multiple learners, and an empirical evidence of the effectiveness of this approach. In this paper we present a brief overview of ensemble methods, explaining the main reasons why they are able to outperform any single classifier within the ensemble, and proposing a taxonomy based on the main ways base classifiers can be generated or combined together.

show abstract

Democracy in neural nets: Voting schemes for classification

Cited by 273 publications

References 9 publications

Detection of Recombination in DNA Multiple Alignments with Hidden Markov Models

Detection of Recombination in DNA Multiple Alignments with Hidden Markov Models

Hierarchical classification of protein function with ensembles of rules and particle swarm optimisation

Ensembles of Learning Machines

Contact Info

Product

Resources

About