Concurrent and predictive validity of dynamic assessments of word reading in young children: A systematic review and meta-analysis

Abstract

Traditional static literacy assessments evaluate acquired knowledge and are prone to floor effects. These tools are also developed almost exclusively for English monolinguals, and therefore cannot be used equitably to evaluate the abilities of bilingual children. Dynamic assessment (DA), which evaluates the ability to learn a skill, is a potential alternative, and more equitable approach to evaluating critical early literacy skills of phonological awareness, sound-symbols knowledge, and decoding. This systematic review and meta-analysis examined the concurrent validity of DAs of early literacy with their static equivalents, and their predictive validity longitudinally with later word reading outcome measures both overall across all populations, and specifically with bilingual and at-risk groups. Thirty studies were identified through searching 5 databases, and the grey literature. Included studies provided a correlation between a dynamic and concurrent static assessment, or a dynamic and a later reading outcome measure. Results of the first random effects meta-analysis suggested that overall, there was a strong relationship between dynamic and static assessments (r=.58). Subgroup analysis revealed that there were significant differences (p=.0012) between DAs of distinct early literacy skills, with decoding (r=.72) and phonological awareness (r=.50) measures demonstrating greater degrees of correlation with their static counterparts, compared to DAs of sound-symbol knowledge (r=.34). The outcomes of the second random effects meta-analysis indicate that there is a similarly strong relationship between DAs and word reading outcome measures overall (r=.58). Subgroup analyses did not reveal significant differences (p=.0593) in the predictive association between DAs of phonological awareness (r=.55) and decoding (r=.58). There were insufficient studies to conduct separate analyses for bilingual and at-risk populations. However, a narrative review suggests that the magnitude of the effect sizes from individual studies conducted with these populations are in line with overall correlational findings. There is also some evidence to suggest that DAs have the capacity to explain a significant amount of variance in later word reading outcomes in bilingual (7-11%) and at-risk groups (7-21%). Future studies should examine the validity of DAs specifically for use with well-defined bilingual and at-risk groups, as these are the populations who potentially have the most to gain from these measures.

Characteristics of dynamic assessments of word reading skills and their implications for validity: A systematic review and meta-analysis

“…The review objectives and meta-analytic approach were planned a priori and detailed in a registered protocol on the Open Science Framework. This protocol is available online at https://osf.io/bcghx/ (Wood & Molnar, 2022).…”

“…Caffrey et al (2008) found that DAs demonstrated greater predictive validity than SA across several domains (e.g., DAs of cognitive ability, literacy, and mathematics). A more recent review focussing exclusively on the domain of word reading skills, reported that DAs of phonological awareness and decoding demonstrate concurrent validity with SAs and predictive validity with word reading outcomes (Wood et al, 2023). In terms of format of DAs of word reading skills, no quantitative comparison has been conducted evaluating differences in associations with word reading outcomes, although findings suggest that there are no differences in classification accuracy of reading disorder for DAs that use a GP vs. TT format (Dixon et al, 2022b)).…”

“…Study inclusion criteria were determined a priori and outlined in the protocol on Open Science Framework (Wood & Molnar, 2022). All studies included in this review were:…”

“…This approach has been shown to reduce bias in testing and misidentification of difficulty because the impact of previous linguistic or educational experiences on test outcomes are minimized (Bedore & Peña, 2008; Petersen & Gillam, 2013). In the domain of word reading skills, DAs have been shown to predict unique variance beyond SAs in reading outcomes (Dixon et al, 2022a), to contribute to the accurate identification of reading difficulties (Dixon et al, 2022b), and to demonstrate strong concurrent validity with equivalent SAs and predictive validity with word reading outcome measures across typically developing, at-risk, bilingual, and monolingual children (Wood et al, 2023). However, these DAs are characterized by substantial heterogeneity in terms of their format, administration method, word, and symbol type.…”

“…Like Caffrey et al, (2008) we will use Pearson’s correlation coefficients as our effect size measure, given that these are the most observed type of effect size reported in studies investigating DA measures. We focus exclusively on DAs of word reading skills: phonological awareness, sound-symbol knowledge, and decoding – as these skills demonstrate concurrent validity with SA counterparts and predictive validity with later word reading outcomes (Wood et al, 2023). In our analyses, we stratify DAs by their format (graduated prompts vs. train/test), their administration method (in-person vs. computer vs. computerized); (also see Dixon et al, 2022a; 2022b), and include stratifications by word type (real word vs. nonword) and symbol type (familiar vs. novel).…”