How causal is a reciprocal effect? Contrasting traditional and new methods to investigate the reciprocal effects model of self-concept and achievement

Abstract

The relationship between students’ subject-specific academic self-concept and their academic achievement is one of the most widely researched topics in educational psychology. A large body of this research has considered cross-lagged panel models (CLPMs), oftentimes synonymously referred to as reciprocal effects models (REMs), as a gold standard to investigate the causal relations between the two variables and has reported evidence for a reciprocal relationship between self-concept and achievement. However, more recent methodological research questioned the plausibility of assumptions that need to be satisfied in order to interpret results from traditional CLPMs causally. In this substantive-methodological synergy, we aimed at contrasting traditional and more recently developed methods to investigate reciprocal effects of students’ academic self-concept and achievement. Specifically, we compared results from CLPMs, from full forward CLPMs (FF-CLPMs), and from random intercept CLPMs (RI-CLPMs) with two weighting approaches developed to study causal effects of continuous treatment variables. To estimate these different models, we used rich longitudinal data of N = 3,757 students from lower secondary schools in Germany. Results from CLPMs, FF-CLPMs, and weighting methods support the reciprocal effects model, particularly when considering math self-concept and grades. In contrast, results from the RI-CLPMs were less consistent. Implications from our study for the interpretation of effects from the different models and methods as well as for school motivation theory are discussed.

Directional Ordering of Self-Concept, School Grades, and Standardized Tests Over Five Years: New Tripartite Models Juxtaposing Within- and Between-Person Perspectives

“…Noting that there are still issues with CLPMs, even with the inclusion of lag-2 effects, they argued for a selection-onobservables CLPM approach based on the observed information in the data (previous measures of the treatment and outcome, and additional covariates), instead of stable trait factors (that are based on modeling assumptions in RI-CLPMs). This approach is consistent with VanderWeele et al's (2020; see also VanderWeele et al, 2016) perspective on causal inference with longitudinal data and has also been recently emphasized by Hübner et al (2022).…”

“…Math self-concept was based on responses to 6 items, but all other constructs were single-item constructs. Excluded in order to avoid clutter are correlated uniquenesses relating responses to the same math self-concept item administered in different years, and correlated residual covariances among the three constructs in Years 6 -9 a few applications of the tripartite RI-CLPMs with latent variables, particularly for REM studies of ASC and achievement (but see Van Lissa et al, 2021;Burns et al, 2019; also see Hübner et al, 2022; we elaborate on the importance of this contribution in Discussion section).…”

“…Thus the selection of covariates is crucial for RI-CLPMs and particular CLPMs. Hence, we find it surprising that this issue has been given limited attention in the design, analysis, and interpretation of these models (see VanderWeele, 2019, for discussion of alternative strategies for selecting covariates; also see Hübner et al, 2022). Cattell's (1966) data cube helps distinguish within-and between-person perspectives.…”

“…However, several researchers recently critiqued considering the RI-CLPM as the default approach to analyzing cross-lagged panel designs (Asendorpf, 2021;Hübner et al, 2022;Lüdtke & Robitzsch, 2021;Orth et al, 2021). More specifically, Lüdtke and Robitzsch (2021) studied the RI-CLPM from a causal inference perspective (e.g., Imbens & Rubin, 2015;Pearl et al, 2016) using mathematical derivation and simulated data.…”

“…Andersen (2021) compares these different modeling approaches and discusses conditions under which they provide similar results (see also Asparouhov & Muthén, 2021). Noting the importance of considering confounding variables in causal interpretations based on CLPMs and RI-CLPMS, Hübner et al, (2022) proposed using weighting strategies that facilitate controlling for a large number of fixed and time-varying covariates. Niepel et al (2021) evaluated CLPM data based on intensive experience sampling of momentary state measures of ASC and achievement.…”