Theoretical Comparison between the Gini Index and Information Gain Criteria

Raileanu, Laura Elena; Stoffel, Kilian

doi:10.1023/b:amai.0000018580.96245.c6

Cited by 549 publications

(253 citation statements)

References 15 publications

Supporting

Mentioning

245

Contrasting

Unclassified

Order By: Relevance

“…These values were accompanied by a label for the correct methylation state reported by the upstream marker on the same DNA strand. The Gini importance was calculated for each feature in a specific context using a forest of extremely randomized trees (21,23).…”

Section: Methodsmentioning

confidence: 99%

Error rates for nanopore discrimination among cytosine, methylcytosine, and hydroxymethylcytosine along individual DNA strands

Schreiber

Wescoe

Abu-Shumays

et al. 2013

Proc. Natl. Acad. Sci. U.S.A.

175

156

View full text Add to dashboard Cite

Cytosine, 5-methylcytosine, and 5-hydroxymethylcytosine were identified during translocation of single DNA template strands through a modified Mycobacterium smegmatis porin A (M2MspA) nanopore under control of phi29 DNA polymerase. This identification was based on three consecutive ionic current states that correspond to passage of modified or unmodified CG dinucleotides and their immediate neighbors through the nanopore limiting aperture. To establish quality scores for these calls, we examined ∼3,300 translocation events for 48 distinct DNA constructs. Each experiment analyzed a mixture of cytosine-, 5-methylcytosine-, and 5-hydroxymethylcytosine-bearing DNA strands that contained a marker that independently established the correct cytosine methylation status at the target CG of each molecule tested. To calculate error rates for these calls, we established decision boundaries using a variety of machine-learning methods. These error rates depended upon the identity of the bases immediately 5′ and 3′ of the targeted CG dinucleotide, and ranged from 1.7% to 12.2% for a single-pass read. We estimate that Q40 values (0.01% error rates) for methylation status calls could be achieved by reading single molecules 5-19 times depending upon sequence context.MspA | epigenetics E pigenetic modifications of DNA help regulate gene transcription in biological cells. In mammals, 5-methylcytosine (mC) modification of CG dinucleotides is known to influence development (1, 2) and contribute to human diseases including cancer (3). Other modifications have been detected at carbon 5 of cytosine including 5-hydroxymethylcytosine (hmC) (4), and more recently 5-formylcytosine, and 5-carboxycytosine (5). Physiological roles for hmC in carcinogenesis and embryonic stem cell differentiation have been proposed (6).High-throughput techniques for mC detection are based on bisulfite treatment of genomic DNA (7). In the conventional assay, cytosine (but not mC nor hmC) is converted to uracil (8). Thus, positions not converted to uracil identify cytosines that were modified in the original genomic sequence. In a landmark paper, Lister et al. (9) used this technique to map genome-wide cytosine methylation in human embryonic stem cells and fetal lung fibroblasts at single-nucleotide precision. Recently, bisulfite strategies for discriminating between mC and hmC using the Tet1 enzyme (10) or by chemical modification of hmC (11) have been described.Single-molecule techniques have emerged as possible alternatives to bisulfite treatment for detecting epigenetic modifications of DNA (12). These single-molecule approaches share several useful features including few processing steps before sequence analysis, long reads that routinely exceed several thousand nucleotides, and the ability to read native DNA strands in heterogeneous mixtures. The most advanced of these single-molecule techniques, from Pacific Biosciences, uses fluorescence to detect labeled nucleotide triphosphates during daughter-strand elongation. This elongation is catalyzed by a DNA pol...

show abstract

Section: Methodsmentioning

confidence: 99%

Error rates for nanopore discrimination among cytosine, methylcytosine, and hydroxymethylcytosine along individual DNA strands

Schreiber

Wescoe

Abu-Shumays

et al. 2013

Proc. Natl. Acad. Sci. U.S.A.

175

156

View full text Add to dashboard Cite

show abstract

“…Gini(S )'s minimum is 0, all of the members in the set belong to the same class, indicating that the maximum useful information can be obtained. When all of the samples in the set distribute equally for each class, Gini(S ) is at its maximum, indicating that the minimum useful information can be obtained [6], [12], [13]. However, most studies of Gini-Index have been used only for splitting attributes in a decision tree.…”

Section: Gini-index Theory For Feature Selectionmentioning

confidence: 99%

“…Chandra et al 2009), among still others [1], [6], [8], [10], [15]. And several researchers have indicated that feature selection was biased towards attributes with a large number of possible values, having more values, a larger number of categories, multiple-valued attributes, a large number of missing values, etc, and many studies on unbiased split selection have been introduced [6]. Recently, Carolin Strobl et al (2007) introduced unbiased split selection for classification trees based on the Gini-Index and a new split selection criterion that avoids variable selection bias on standard impurity measures, and Marco Sandri (2008) presented a simple and effective method for bias correction focused on the easily generalizable case of the Gini-Index [3], [7].…”

Section: Introductionmentioning

confidence: 98%

“…The recent typical studies on the Gini-Index have concerned a formal methodology for comparing multiple split criteria and a formal description of how to theoretically select between split criteria , and a fuzzy decision tree algorithm GiniIndex (B. Chandra et al 2009), among still others [1], [6], [8], [10], [15]. And several researchers have indicated that feature selection was biased towards attributes with a large number of possible values, having more values, a larger number of categories, multiple-valued attributes, a large number of missing values, etc, and many studies on unbiased split selection have been introduced [6].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Improved Gini-Index Algorithm to Correct Feature-Selection Bias in Text Classification

Park

Kwon

2011

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

Heum PARK†a) , Nonmember and Hyuk-Chul KWON †b) , Member SUMMARY This paper presents an improved Gini-Index algorithm to correct feature-selection bias in text classification. Gini-Index has been used as a split measure for choosing the most appropriate splitting attribute in decision tree. Recently, an improved Gini-Index algorithm for feature selection, designed for text categorization and based on Gini-Index theory, was introduced, and it has proved to be better than the other methods. However, we found that the Gini-Index still shows a feature selection bias in text classification, specifically for unbalanced datasets having a huge number of features. The feature selection bias of the Gini-Index in feature selection is shown in three ways: 1) the Gini values of low-frequency features are low (on purity measure) overall, irrespective of the distribution of features among classes, 2) for high-frequency features, the Gini values are always relatively high and 3) for specific features belonging to large classes, the Gini values are relatively lower than those belonging to small classes. Therefore, to correct that bias and improve feature selection in text classification using Gini-Index, we propose an improved Gini-Index (I-GI) algorithm with three reformulated Gini-Index expressions. In the present study, we used global dimensionality reduction (DR) and local DR to measure the goodness of features in feature selections. In experimental results for the I-GI algorithm, we obtained unbiased feature values and eliminated many irrelevant general features while retaining many specific features. Furthermore, we could improve the overall classification performances when we used the local DR method. The total averages of the classification performance were increased by 19.4 %, 15.9 %, 3.3 %, 2.8 % and 2.9 % (kNN) in Micro-F1, 14 %, 9.8 %, 9.2 %, 3.5 % and 4.3 % (SVM) in Micro-F1, 20 %, 16.9 %, 2.8 %, 3.6 % and 3.1 % (kNN) in Macro-F1, 16.3 %, 14 %, 7.1 %, 4.4 %, 6.3 % (SVM) in Macro-F1, compared with tf*idf, χ 2 , Information Gain, Odds Ratio and the existing Gini-Index methods according to each classifier.

show abstract

“…The advantage of such an approach, while it can be computationally more expensive, is that it captures more information about the cluster. This is akin to classification with the Gini index or information gain heuristics [16], which summarize a set of examples by means of its class distribution instead of the majority class. We plan to incorporate different types of constraints in our models.…”

Section: Future Workmentioning

confidence: 99%

Analysis of Time Series Data with Predictive Clustering Trees

Džeroski

Gjorgjioski

Slavkov

et al.

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Predictive clustering is a general framework that unifies clustering and prediction. This paper investigates how to apply this framework to cluster time series data. The resulting system, Clus-TS, constructs predictive clustering trees (PCTs) that partition a given set of time series into homogeneous clusters. In addition, PCTs provide a symbolic description of the clusters. We evaluate Clus-TS on time series data from microarray experiments. Each data set records the change over time in the expression level of yeast genes in response to a change in environmental conditions. Our evaluation shows that Clus-TS is able to cluster genes with similar responses, and to predict the time series based on the description of a gene. Clus-TS is part of a larger project where the goal is to investigate how global models can be combined with inductive databases.

show abstract

Theoretical Comparison between the Gini Index and Information Gain Criteria

Cited by 549 publications

References 15 publications

Error rates for nanopore discrimination among cytosine, methylcytosine, and hydroxymethylcytosine along individual DNA strands

Error rates for nanopore discrimination among cytosine, methylcytosine, and hydroxymethylcytosine along individual DNA strands

Improved Gini-Index Algorithm to Correct Feature-Selection Bias in Text Classification

Analysis of Time Series Data with Predictive Clustering Trees

Contact Info

Product

Resources

About