Inter-node Hellinger Distance based Decision Tree

Akash, Pritom Saha; Kadir, Md. Eusha; Ali, Amin Ahsan; Shoyaib, Mohammad

doi:10.24963/ijcai.2019/272

Cited by 12 publications

(6 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Motivated by HDT, several extensions have been proposed to handle different tasks under class imbalance: data streams (Grzyb, Klikowski, & Woźniak, 2021;Lyon et al, 2014), multilabel classification (Daniels & Metaxas, 2017), and multiclass classification (Hoens, Qian, Chawla, & Zhou, 2012). Other works have used the Hellinger distance to propose their own methods that aim to outperform the HDT (Akash, Kadir, Ali, & Shoyaib, 2019;Su & Cao, 2019). Despite the popularity of HDT in other domains, it is not yet explored in weakly supervised learning.…”

Section: Imbalanced Learningmentioning

confidence: 99%

Hellinger distance decision trees for PU learning in imbalanced data sets

2023

View full text Add to dashboard Cite

Learning from positive and unlabeled data, or PU learning, is the setting in which a binary classifier can only train from positive and unlabeled instances, the latter containing both positive as well as negative instances. Many PU applications, e.g., fraud detection, are also characterized by class imbalance, which creates a challenging setting. Not only are fewer minority class examples compared to the case where all labels are known, there is also only a small fraction of unlabeled observations that would actually be positive. Despite the relevance of the topic, only a few studies have considered a class imbalance setting in PU learning. In this paper, we propose a novel technique that can directly handle imbalanced PU data, named the PU Hellinger Decision Tree (PU-HDT). Our technique exploits the class prior to estimate the counts of positives and negatives in every node in the tree. Moreover, the Hellinger distance is used instead of more conventional splitting criteria because it has been shown to be class-imbalance insensitive. This simple yet effective adaptation allows PU-HDT to perform well in highly imbalanced PU data sets. We also introduce PU Stratified Hellinger Random Forest (PU-SHRF), which uses PU-HDT as its base learner and integrates a stratified bootstrap sampling. Our empirical analysis shows that PU-SHRF substantially outperforms state-of-the-art PU learning methods for imbalanced data sets in most experimental settings.

show abstract

Section: Imbalanced Learningmentioning

confidence: 99%

Hellinger distance decision trees for PU learning in imbalanced data sets

2023

View full text Add to dashboard Cite

show abstract

“…Akash et al [13] proposed a skew insensitive splitting criterion called Inter-node Hellinger Distance (iHD) that uses the squared Hillinger distance (D 2 H ) to measure the heterogeneity between the class probability distributions of the parent and child nodes.…”

Section: Inter-node Hellinger Distancementioning

confidence: 99%

“…Breiman and Friedman [2] introduced Classification And Regression Trees (CART) which provides the Gini Index and towing criterion as impurity measurement. More recently, Akash et al [13] proposed a splitting criterion called Inter-node Hellinger Distance (iHD), which measures the distance between parent and children nodes using Hellinger Distance. The splitting criteria of the above described and our proposal algorithms are compared in Table I below.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Novel Splitting Criterion Inspired by Geometric Mean Metric Learning for Decision Tree

Li¹,

Chen²

2022

Preprint

View full text Add to dashboard Cite

Decision tree (DT) attracts persistent research attention due to its impressive empirical performance and interpretability in numerous applications. However, the growth of traditional yet widely-used univariate decision trees (UDTs) is quite time-consuming as they need to traverse all the features to find the splitting value with the maximal reduction of the impurity at each internal node. In this paper, we newly design a splitting criterion to speed up the growth. The criterion is induced from Geometric Mean Metric Learning (GMML) and then optimized under its diagonalized metric matrix constraint, consequently, a closed-form rank of feature discriminant abilities can at once be obtained and the top 1 feature at each node used to grow an intent DT (called as dGMML-DT, where d is an abbreviation for diagonalization). We evaluated the performance of the proposed methods and their corresponding ensembles on benchmark datasets. The experiment shows that dGMML-DT achieves comparable or better classification results more efficiently than the UDTs with 10x average speedup. Furthermore, dGMML-DT can straightforwardly be extended to its multivariable counterpart (dGMML-MDT) without needing laborious operations.

show abstract

“…In contrast, algorithm oriented approaches are the modifications of traditional algorithms such as DT and kNN. The modified DTs for imbalanced classification are Hellinger Distance DT (HDDT) [5], Class Confidence Proportion DT (CCPDT) [13] and Weighted Inter-node Hellinger Distance DT (iHDwDT) [1]. These DTs use different splitting criteria while selecting a feature in split point.…”

Section: Introductionmentioning

confidence: 99%

A Proximity Weighted Evidential k Nearest Neighbor Classifier for Imbalanced Data

Kadir

Akash

Sharmin

et al. 2020

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

In k Nearest Neighbor (kNN) classifier, a query instance is classified based on the most frequent class of its nearest neighbors among the training instances. In imbalanced datasets, kNN becomes biased towards the majority instances of the training space. To solve this problem, we propose a method called Proximity weighted Evidential kNN classifier. In this method, each neighbor of a query instance is considered as a piece of evidence from which we calculate the probability of class label given feature values to provide more preference to the minority instances. This is then discounted by the proximity of the neighbor to prioritize the closer instances in the local neighborhood. These evidences are then combined using Dempster-Shafer theory of evidence. A rigorous experiment over 30 benchmark imbalanced datasets shows that our method performs better compared to 12 popular methods. In pairwise comparison of these 12 methods with our method, in the best case, our method wins in 29 datasets, and in the worst case it wins in least 19 datasets. More importantly, according to Friedman test the proposed method ranks higher than all other methods in terms of AUC at 5% level of significance.

show abstract

Inter-node Hellinger Distance based Decision Tree

Cited by 12 publications

References 8 publications

Hellinger distance decision trees for PU learning in imbalanced data sets

Hellinger distance decision trees for PU learning in imbalanced data sets

A Novel Splitting Criterion Inspired by Geometric Mean Metric Learning for Decision Tree

A Proximity Weighted Evidential k Nearest Neighbor Classifier for Imbalanced Data

Contact Info

Product

Resources

About