Group membership prediction when known groups consist of unknown subgroups: a Monte Carlo comparison of methods

Abstract

Classification using standard statistical methods such as linear discriminant analysis (LDA) or logistic regression (LR) presume knowledge of group membership prior to the development of an algorithm for prediction. However, in many real world applications members of the same nominal group, might in fact come from different subpopulations on the underlying construct. For example, individuals diagnosed with depression will not all have the same levels of this disorder, though for the purposes of LDA or LR they will be treated in the same manner. The goal of this simulation study was to examine the performance of several methods for group classification in the case where within group membership was not homogeneous. For example, suppose there are 3 known groups but within each group two unknown classes. Several approaches were compared, including LDA, LR, classification and regression trees (CART), generalized additive models (GAM), and mixture discriminant analysis (MIXDA). Results of the study indicated that CART and mixture discriminant analysis were the most effective tools for situations in which known groups were not homogeneous, whereas LDA, LR, and GAM had the highest rates of misclassification. Implications of these results for theory and practice are discussed.

Rapid Automatized Naming and Explicit Phonological Processing in Children With Developmental Dyslexia: A Study With Portuguese-Speaking Children in Brazil

“…Such is the case here: within the group of children with dyslexia, there exist differentiated levels of actual reading deficit severity. In a recent simulation study, more traditional methods for classification, such as linear discriminant analysis or logistic regression, have been shown not to perform well in the presence of known group mixtures, and CART was identified as the method that provided the most accurate predictions in such situation (Holmes Finch et al, 2014).…”

“…We compared the performance of children with dyslexia to a chronological age-matched control group of typical readers, and we used CART analysis to identify potential variables as significant predictors of dyslexia diagnosis. This statistical technique has proven to be an effective classification tool especially in situations where classification groups are not homogeneous (Holmes Finch et al, 2014).…”

Daily Links Between Helping Behaviors and Emotional Well‐Being During Late Adolescence

“…We person-centered all level-1 predictors, and we included on the intercept person-mean values for each of our daily predictors (i.e., each helping behavior; Curran & Bauer, 2011). This statistical approach accounts for dependency within participants and introduces less bias compared to traditional statistical analyses, such as repeated measures analysis of variance (Finch, Bolin, & Kelley, 2014;Raudenbush & Bryk, 2002). To increase the robustness of our findings, we additionally controlled for previous day levels of positive and negative affect (i.e., to test if helping behaviors were associated with positive and negative affect over and above prior day levels).…”

A Case for Multisite Second Language Acquisition Research: Challenges, Risks, and Rewards

“…Even when present at low levels, such intraclass correlations underestimate the standard errors for model parameters. This can lead to Type I error (false positives for statistical significance) if nesting is ignored in data analysis (Bickel, 2007; Finch, Bolin, & Kelley, 2014).…”