In recent studies, the existence and relevance of achievement composition effects on students' individual achievement have been called into question because of the methodological challenges arising in multilevel analyses. Our study examined how class-average achievement is related to students' achievement development across one school year. We used data from Germany, which has a secondary school system with large achievement differences between schools and classrooms due to rigid, explicit betweenschool tracking practices. We accounted for two methodological challenges, controlling for both selection bias and measurement error. Adopting an approach based on integrative data analysis (IDA), we systematically (re)analyzed five German longitudinal large-scale data sets. This IDA approach allowed us to quantify the extent to which results vary across (a) different longitudinal data sets and (b) different analytical strategies (i.e., ways of accounting for confounding variables and measurement reliability). Overall, we found both general achievement composition effects and narrower peer spillover effects (i.e., effects of student composition above and beyond the effects of tracking) in the German setting, even after controlling for measurement error and selection bias. Our results counter recent suggestions that composition effects on achievement development may be mere phantom effects due to methodological misspecifications. However, estimates of composition effects varied substantially based on the analytical approach. We conclude with considerations regarding how to interpret composition effects in multilevel modeling and which effects are of interest for educational research.
Educational Impact and Implications StatementThe study investigates whether students experience better achievement development when they have high-achieving classmates, an effect that is known as the achievement composition effect. This question has been controversial in the literature, and several researchers who used a new analytical strategy to analyze compositional effects have recently suggested that achievement composition effects may be mere "phantom effects," meaning that being in a classroom with higher-achieving peers may not have any beneficial effect on students' achievement development or may even have detrimental effects on it. We (re)analyzed five longitudinal studies from Germany, which allowed us to examine the extent to which results varied across studies using different longitudinal data sets and different analytical strategies. Overall, we found positive achievement composition effects. However, they differed based on the analytical strategies used and how they operationalized the effect. The results support the argument that being surrounded by higher-achieving peers can be beneficial for students' learning gains and, at the same time, being surrounded by lower-achieving peers may negatively affect students' learning gains.