Untitled

Abstract

Abstract. The simple Bayesian classifier is known to be optimal when attributes are independent given the class, but the question of whether other sufficient conditions for its optimality exist has so far not been explored. Empirical results showing that it performs surprisingly well in many domains containing clear attribute dependences suggest that the answer to this question may be positive. This article shows that, although the Bayesian classifier's probability estimates are only optimal under quadratic loss if the independence assumption holds, the classifier itself can be optimal under zero-one loss (misclassification rate) even when this assumption is violated by a wide margin. The region of quadratic-loss optimality of the Bayesian classifier is in fact a second-order infinitesimal fraction of the region of zero-one optimality. This implies that the Bayesian classifier has a much greater range of applicability than previously thought. For example, in this article it is shown to be optimal for learning conjunctions and disjunctions, even though they violate the independence assumption. Further, studies in artificial domains show that it will often outperform more powerful classifiers for common training set sizes and numbers of attributes, even if its bias is a priori much less appropriate to the domain. This article's results also imply that detecting attribute dependence is not necessarily the best way to extend the Bayesian classifier, and this is also verified empirically.

Evaluation of machine-learning methods for ligand-based virtual screening

“…Although perhaps not recognised when the approach was first introduced, substructural analysis is an example of a naive Bayesian classifier (NBC) [17,63,64]. An NBC is a simple classification algorithm that is based on the use of Bayes' theorem and on strong assumptions as to the statistical independence of the descriptors characterising the objects that are to be classified.…”

“…An NBC is a simple classification algorithm that is based on the use of Bayes' theorem and on strong assumptions as to the statistical independence of the descriptors characterising the objects that are to be classified. The use of "naive" arises from the independence assumptions: these are often demonstrably incorrect, but this has not seemed to affect the performance of the classifier in many application domains [63].…”

Prediction of Outcome in the Vegetative State by Machine Learning Algorithms: A Model for Clinicians?

“…Four different machine learning methods were used for data analysis: K Nearest Neighbours (K-NN) [15], Naïve Bayes [16], C4.5 Decision Tree [17,18], Support Vector Machine (SVM) [19]. The selected algorithms follow distinct approaches in model development.…”

Predicción de la epidemia del virus respiratorio sincitial en Bogotá D.C. utilizando variables climatológicas

“…Sin embargo, cuando se trata de problemas reales, muchas veces esta suposición no se cumple. En algunas ocasiones, no obstante, el paradigma bayesiano ingenuo ha demostrado un buen funcionamiento (12,13). El clasificador bayesiano ingenuo aplica el teorema de Bayes con el fin de predecir para cada instancia x la clase c∈C, con la máxima probabilidad a posteriori.…”