Attribute Value Weighted Average of One-Dependence Estimators

Abstract

Abstract:Of numerous proposals to improve the accuracy of naive Bayes by weakening its attribute independence assumption, semi-naive Bayesian classifiers which utilize one-dependence estimators (ODEs) have been shown to be able to approximate the ground-truth attribute dependencies; meanwhile, the probability estimation in ODEs is effective, thus leading to excellent performance. In previous studies, ODEs were exploited directly in a simple way. For example, averaged one-dependence estimators (AODE) weaken the attribute independence assumption by directly averaging all of a constrained class of classifiers. However, all one-dependence estimators in AODE have the same weights and are treated equally. In this study, we propose a new paradigm based on a simple, efficient, and effective attribute value weighting approach, called attribute value weighted average of one-dependence estimators (AVWAODE). AVWAODE assigns discriminative weights to different ODEs by computing the correlation between the different root attribute value and the class. Our approach uses two different attribute value weighting measures: the Kullback-Leibler (KL) measure and the information gain (IG) measure, and thus two different versions are created, which are simply denoted by AVWAODE-KL and AVWAODE-IG, respectively. We experimentally tested them using a collection of 36 University of California at Irvine (UCI) datasets and found that they both achieved better performance than some other state-of-the-art Bayesian classifiers used for comparison.

Application of Machine Learning Techniques for Clinical Predictive Modeling: A Cross-Sectional Study on Nonalcoholic Fatty Liver Disease in China

“…The ensemble classification algorithms include adaptive boosting (AdaBoost) [ 30 ], bootstrap aggregating (bagging) [ 31 ], and random forest [ 32 , 33 ]. The algorithm extensions include hidden naïve Bayes (HNB) and aggregating one-dependence estimators (AODE) [ 19 – 22 ].…”

“…In principle, ML has four steps: problem definition, data collection and preparation, model building, and model prediction. There are 11 state-of-the-art machine learning techniques [ 18 – 22 ], namely, logistic regression (LR), k-nearest neighbor (kNN), support vector machine (SVM), naïve Bayes (NB), Bayesian network (BN), decision tree (C4.5), AdaBoosting, bagging, random forest (RF), hidden naïve Bayes (HNB), and aggregating one-dependence estimators (AODE).…”

Self-Adaptive Attribute Value Weighting for Averaged One-Dependence Estimators

“…Bayesian network (BN) [1]- [4] provides a powerful tool for knowledge representation and inference under conditions of uncertainty. Since the 1990s, the study of Bayesian network classifier (BNC) for classification has attracted tremendous attention after the success of Naive Bayes (NB) [5]- [8].…”

“…• Model weighting [4], [17] computes the weight associated with each SPODE to combine their probability estimates linearly.…”

“…We argue that overfitting to the testing instance will help improve rather than harm the generalization performance, and the significance of each SPODE should vary while classifying different instances, especially for highly predictive SPODEs. Yu et al [4] considered the specific characteristics of each testing instance and adjusted the weights to different SPODEs adaptively by computing the correlation between the root attribute value and the class. However, the notion that the weight of each SPODE is irrelevant to non-root attribute values is not convincing.…”