Reliability Parameters to Improve Combination Strategies in Multi-Expert Systems

Cordella, Luigi P.; Foggia, Pasquale; Sansone, Carlo; Tortorella, Francesco; Vento, Mario

doi:10.1007/s100440050029

Cited by 76 publications

(64 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The reliability-based weighted voting (RBWV) introduced in [4] is another example of dynamic voting. It uses a model-dependent estimation of the reliability (confidence) of predictions for each particular instance instead of local accuracy as the weights.…”

Section: Discussionmentioning

confidence: 99%

Dynamic integration of classifiers for handling concept drift

et al. 2008

View full text Add to dashboard Cite

Abstract. In the real world concepts are often not stable but change with time. A typical example of this in the biomedical context is antibiotic resistance, where pathogen sensitivity may change over time as new pathogen strains develop resistance to antibiotics that were previously effective. This problem, known as concept drift, complicates the task of learning a model from data and requires special approaches, different from commonly used techniques that treat arriving instances as equally important contributors to the final concept. The underlying data distribution may change as well, making previously built models useless. This is known as virtual concept drift. Both types of concept drifts make regular updates of the model necessary. Among the most popular and effective approaches to handle concept drift is ensemble learning, where a set of models built over different time periods is maintained and the best model is selected or the predictions of models are combined, usually according to their expertise level regarding the current concept. In this paper we propose the use of an ensemble integration technique that would help to better handle concept drift at an instance level. In dynamic integration of classifiers, each base classifier is given a weight proportional to its local accuracy with regard to the instance tested, and the best base classifier is selected, or the classifiers are integrated using weighted voting. Our experiments with synthetic data sets simulating abrupt and gradual concept drifts and with a real-world antibiotic resistance data set demonstrate that dynamic integration of classifiers built over small time intervals or fixed-sized data blocks can be significantly better than majority voting and weighted voting, which are currently the most commonly used integration techniques for handling concept drift with ensembles.

show abstract

Section: Discussionmentioning

confidence: 99%

Dynamic integration of classifiers for handling concept drift

et al. 2008

View full text Add to dashboard Cite

show abstract

“…kN N is a statistical classifier where its classification reliability is computed using the definition reported in [14]: O min is the distance between x and the nearest sample of the validation set, i.e. the sample determining the class, O max is the highest among the values of O min obtained from all samples of the output class belonging to the test set, and O min2 is the distance between x and the nearest sample in the validation set belonging to a class other than the output one.…”

Section: Classifier Reliabilitymentioning

confidence: 99%

Biomedical Images Classification by Universal Nearest Neighbours Classifier Using Posterior Probability

D'Ambrosio

Ali

Nock

et al. 2012

Machine Learning in Medical Imaging

View full text Add to dashboard Cite

Abstract. Universal Nearest Neighbours (unn) is a classifier recently proposed, which can also effectively estimates the posterior probability of each classification act. This algorithm, intrinsically binary, requires the use of a decomposition method to cope with multiclass problems, thus reducing their complexity in less complex binary subtasks. Then, a reconstruction rule provides the final classification. In this paper we show that the application of unn algorithm in conjunction with a reconstruction rule based on the posterior probabilities provides a classification scheme robust among different biomedical image datasets. To this aim, we compare unn performance with those achieved by Support Vector Machine with two different kernels and by a k Nearest Neighbours classifier, and applying two different reconstruction rules for each of the aforementioned classification paradigms. The results on one private and five public biomedical datasets show satisfactory performance.

show abstract

“…The idea is based on the widely accepted result that a MES approach generally produces better results than those obtained by individual composing experts, since different features and recognition systems complement each other in classification performance. Indeed, the MES takes advantage of the strengths of the single experts, without being affected by their weaknesses [16,17]. Furthermore, constructing balanced subsets of the original TS avoids the drawbacks of under and oversampling.…”

Section: Techniques For Handling Imbalancedmentioning

confidence: 99%

“…Reliability Estimation. Exploiting information derived from classifier output permits to properly estimates the reliability of the decision of each classification act [16,17]. Reliability takes into account the many issues that influence the achievement of a correct classification, such as the noise affecting the samples domain or the difference between the objects to be recognized and those used to train the classifier.…”

Section: Techniques For Handling Imbalancedmentioning

confidence: 99%

A Hybrid Approach Handling Imbalanced Datasets

Soda

2009

Image Analysis and Processing – ICIAP 2009

View full text Add to dashboard Cite

Abstract. Several binary classification problems exhibit imbalance in class distribution, influencing system learning. Indeed, traditional machine learning algorithms are biased towards the majority class, thus producing poor predictive accuracy over the minority one. To overcome this limitation, many approaches have been proposed up to now to build artificially balanced training sets. Further to their specific drawbacks, they achieve more balanced accuracies on each class harming the global accuracy. This paper first reviews the more recent method coping with imbalanced datasets and then proposes a strategy overcoming the main drawbacks of existing approaches. It is based on an ensemble of classifiers trained on balanced subsets of the original imbalanced training set working in conjunction with the classifier trained on the original imbalanced dataset. The performance of the method has been estimated on six public datasets, proving its effectiveness also in comparison with other approaches. It also gives the chance to modify the system behaviour according to the operating scenario.

show abstract

Reliability Parameters to Improve Combination Strategies in Multi-Expert Systems

Cited by 76 publications

References 12 publications

Dynamic integration of classifiers for handling concept drift

Dynamic integration of classifiers for handling concept drift

Biomedical Images Classification by Universal Nearest Neighbours Classifier Using Posterior Probability

A Hybrid Approach Handling Imbalanced Datasets

Contact Info

Product

Resources

About