The Effects of Initially Misclassified Data on the Effectiveness of Discriminant Function Analysis and Finite Mixture Modeling

Holden, Jocelyn; Kelley, Ken

doi:10.1177/0013164409344533

Cited by 9 publications

(13 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, several studies have shown that systematic misclassification in the two group case is detrimental to classification accuracy of traditional supervised classification methods (i.e., Discriminant analysis) (Lachenbruch, 1966, 1974, 1979; McLachlan, 1972; Chhikara and McKeon, 1984; Holden and Kelley, 2010). There is little research, however, investigating the impact of systematic training data misclassification when three true groups are present, or for newer classification and data mining techniques.…”

Section: Discussion Of Prediction Methodsmentioning

confidence: 99%

“…As has been demonstrated previously, statistical classification methods are greatly affected by the characteristics of the data under study. Previous research indicates classification accuracy generally increases with increased sample size (Holden and Kelley, 2010 ; Holden et al, 2011 ; Pai et al, 2012 ), discrepancy in group size (Lei and Koehly, 2003 ; deCraen et al, 2006 ; Holden and Kelley, 2010 ; Holden et al, 2011 ), group separation (Blashfield, 1976 ; Lei and Koehly, 2003 ; Holden and Kelley, 2010 ; Holden et al, 2011 ), and number of variables used in the classification (Breckenridge, 2000 ). Assumption violations (Lei and Koehly, 2003 ; Rausch and Kelley, 2009 ), outliers and presence of multicollinearity (Pai et al, 2012 ) generally lead to decreased classification accuracy.…”

Section: Introductionmentioning

confidence: 96%

See 1 more Smart Citation

Supervised classification in the presence of misclassified training data: a Monte Carlo simulation study in the three group case

Bolin

Finch

2014

Front. Psychol.

View full text Add to dashboard Cite

Statistical classification of phenomena into observed groups is very common in the social and behavioral sciences. Statistical classification methods, however, are affected by the characteristics of the data under study. Statistical classification can be further complicated by initial misclassification of the observed groups. The purpose of this study is to investigate the impact of initial training data misclassification on several statistical classification and data mining techniques. Misclassification conditions in the three group case will be simulated and results will be presented in terms of overall as well as subgroup classification accuracy. Results show decreased classification accuracy as sample size, group separation and group size ratio decrease and as misclassification percentage increases with random forests demonstrating the highest accuracy across conditions.

show abstract

Section: Discussion Of Prediction Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 96%

Supervised classification in the presence of misclassified training data: a Monte Carlo simulation study in the three group case

Bolin

Finch

2014

Front. Psychol.

View full text Add to dashboard Cite

show abstract

“…However, when the accuracy of the training data is in question, the accuracy of supervised classification methods is debatable. Authors have previously discussed the accuracy of classification with misclassified training data for discriminant function methods and LR (Chhikara & McKeon, 1984;Grayson, 1987;Holden & Kelley, 2010;Lachenbruch, 1966Lachenbruch, , 1974Lachenbruch, , 1979Lei & Koehly, 2003;McLachlan, 1972); however, the topic of training data misclassification has yet to be studied for these newer classification methods. The results of the current study indicate that alternative methods of classification, particularly CART, may provide better classification accuracy, especially for complex models.…”

Section: Directions For Future Researchmentioning

confidence: 99%

A Comparison of Two-Group Classification Methods

Holden

Finch

Kelley

2011

Educational and Psychological Measurement

Self Cite

View full text Add to dashboard Cite

The statistical classification of N individuals into G mutually exclusive groups when the actual group membership is unknown is common in the social and behavioral sciences. The results of such classification methods often have important consequences. Among the most common methods of statistical classification are linear discriminant analysis, quadratic discriminant analysis, and logistic regression. However, recent developments in the statistics literature have brought new and potentially more flexible classification models to the forefront. Although these new models are increasingly being used in the physical sciences and marketing research, they are still relatively little used in the social and behavioral sciences. The purpose of this article is to provide a comparison of these modern methods with the classical methods widely used in situations that are relevant in the social and behavioral sciences. This study uses a large-scale Monte Carlo simulation study for the comparisons, as analytic comparisons are often not tractable. Results indicate that classification and regression trees generally produced the highest classification accuracy of all techniques tested, though study design characteristics such as sample size and model complexity can greatly influence optimal choice or effectiveness of statistical classification method.

show abstract

“…In the unequal scenario for two groups, the ratio was 75/25, while with three groups the unequal ratio was 60/20/20. Previous research (e.g., Holden and Kelley, 2010 ; Holden et al, 2011 ) found that unequal group sizes had an impact on performance of group prediction methods, and therefore was included in the current study.…”

Section: Methodsmentioning

confidence: 99%

Group membership prediction when known groups consist of unknown subgroups: a Monte Carlo comparison of methods

Finch¹,

Bolin²,

Kelley³

2014

Front. Psychol.

Self Cite

View full text Add to dashboard Cite

Classification using standard statistical methods such as linear discriminant analysis (LDA) or logistic regression (LR) presume knowledge of group membership prior to the development of an algorithm for prediction. However, in many real world applications members of the same nominal group, might in fact come from different subpopulations on the underlying construct. For example, individuals diagnosed with depression will not all have the same levels of this disorder, though for the purposes of LDA or LR they will be treated in the same manner. The goal of this simulation study was to examine the performance of several methods for group classification in the case where within group membership was not homogeneous. For example, suppose there are 3 known groups but within each group two unknown classes. Several approaches were compared, including LDA, LR, classification and regression trees (CART), generalized additive models (GAM), and mixture discriminant analysis (MIXDA). Results of the study indicated that CART and mixture discriminant analysis were the most effective tools for situations in which known groups were not homogeneous, whereas LDA, LR, and GAM had the highest rates of misclassification. Implications of these results for theory and practice are discussed.

show abstract

The Effects of Initially Misclassified Data on the Effectiveness of Discriminant Function Analysis and Finite Mixture Modeling

Cited by 9 publications

References 34 publications

Supervised classification in the presence of misclassified training data: a Monte Carlo simulation study in the three group case

Supervised classification in the presence of misclassified training data: a Monte Carlo simulation study in the three group case

A Comparison of Two-Group Classification Methods

Group membership prediction when known groups consist of unknown subgroups: a Monte Carlo comparison of methods

Contact Info

Product

Resources

About