A framework of R-squared measures for single-level and multilevel regression mixture models.

Abstract

Psychologists commonly apply regression mixture models in single-level (i.e., unclustered) and multilevel (i.e., clustered) data analysis contexts. Though researchers applying nonmixture regression models typically report R-squared measures of explained variance, there has been no general treatment of R-squared measures for single-level and multilevel regression mixtures. Consequently, it is common for researchers to summarize results of a fitted regression mixture by simply reporting class-specific regression coefficients and their associated p values, rather than considering measures of effect size. In this article, we fill this gap by providing an integrative framework of R-squared measures for single-level regression mixture models and multilevel regression mixture models (with classes at Level-2 or both levels). Specifically, we describe 11 R-squared measures that are distinguished based on what the researcher chooses to consider as outcome variance and what sources the researcher chooses to contribute to predicted variance. We relate these measures analytically and through simulated graphical illustrations. Further, we demonstrate how these R-squared measures can be decomposed in novel ways into substantively meaningful sources of explained variance. We describe and demonstrate new software tools to allow researchers to compute these R-squared measures and decompositions in practice. Using 2 empirical examples, we show how researchers can answer distinct substantive questions with each measure and can gain insights by interpreting the set of measures in juxtaposition to each other. (PsycINFO Database Record

Development of a Rapid Point-Of-Care Test that Measures Neutralizing Antibodies to SARS-CoV-2

“…Importantly, LFA accounts for 52% of observed IC 50 variance ( R 2 = 0.5187) while, in comparison, Ortho Vitros Anti-SARS-CoV-2 IgG test accounts for 27% ( R 2 = 0.2725). Although absolute quantitation of a construct demands an excellent coefficient of determination ( R 2 ≥ 0.99)(33), variables with R 2 ≥ 0.5 are highly predictive in univariate regression models while measures with R 2 < 0.5 are recommended for use in multivariate models in combination with complementary measures to increase predictive accuracy(34, 35). Additionally, Bland-Altman analysis ( Figure 5 ) showed the Ortho Vitros Anti-SARS-CoV-2 IgG test to be prone to underestimation of IC 50 values.…”

Confirmation Bias and the Persistence of Misinformation on Climate Change

“…Age, sex, and level of education were entered as controlled covariates in all models. Effect size estimates (i.e., η 2 ) from multilevel modeling were calculated as the proportion of variance explained following the procedure in Rights and Sterba (2018). Multilevel SEM models were estimated in LISREL 8.80 to test the proposed conceptual model with individuals being the clustering factor.…”

Quantifying explained variance in multilevel models: An integrative framework for defining R-squared measures.

“…Hence, we do not recommend reporting just one combined-source measure in isolation (for reasons illustrated graphically in a later section, entitled Limitations of the Common Practice of Only Reporting a Single MLM R 2 ). Rather, we encourage researchers to interpret a given measure in juxtaposition to other measures within the context of the full decomposition (relatedly, see Rights & Sterba, 2017).…”

“…For instance, LaHuis et al (2014) emphasize that “explained variance measures provide a useful summary of the magnitude of effects and may be particularly useful in multilevel studies where unstandardized coefficients are reported often” (p. 446). In general, R 2 ’s indicate the proportion of variance explained by the model and, as such, are considered measures of effect size 1 that (a) describe the correspondence between a model’s predictions and the observed data (such that higher values of an R 2 measure mean predicted outcomes are more similar to the actual outcomes); (b) have an intuitive metric with well-defined endpoints (0 and 1); and (c) can be compared across studies with similar designs (e.g., Gelman & Hill, 2007; Kvålseth, 1985; Rights & Sterba, 2017; Xu, 2003). As summarized by Roberts et al (2011) “With the further encouragement from editors to begin reporting effect sizes in all research, it is becoming more necessary for researchers using MLM to be able to explain their results in a way that is common with other statistical methods” (p. 229).…”