The Cognitive Validity of Child English Language Tests: What Young Language Learners and Their Native‐Speaking Peers Can Reveal

Abstract

This study investigated the cognitive validity of two child English language tests. Some teachers maintain that these types of tests may be cognitively invalid because native‐English‐speaking children would not do well on them (Winke, 2011). So the researchers had native speakers and learners of English aged 7 to 9 take sample versions of two standardized English reading and writing tests: the Young Learners Tests of English, Bronze and Silver, administered by Cambridge Michigan Language Assessments. They videotaped the children taking the tests, had them draw pictures of how they felt during testing, and interviewed them. The tests reliably discriminated learners from native speakers. However, 3 of 25 items on the Bronze test and 5 of 40 on the Silver test proved more difficult for native speakers than for language learners. The researchers interviewed the children to uncover why; incorrect responses stemmed from lack of assessment literacy or age‐related cognitive limitations, not deficits in English. The researchers discuss whether this is a problem and conclude that standardized language tests for children, even those already psychometrically reliable and valid, can be improved upon by interviewing child test takers. They stress that parallel information from same‐age, native‐English‐speaking peers is informative in revealing construct‐irrelevant variance.

Does younger mean better? Age of onset, learning rate and shortterm L2 proficiency in young Danish learners of English

“…All three tests presented the test items via tape recordings. Listening to an unfamiliar voice on tape may add a cognitive load that could affect children's linguistic performance differently at different ages (e.g., Field, 2018;Winke et al, 2018). However, on the PPVT and the TROG, results showed that the ES had already at wave 2 caught up with the LS' scores from wave 1 so the presentation method does not appear to bias the results of the ES.…”

Development and validation of a Chinese character acquisition assessment for second-language kindergarteners

“…Lastly, future research should establish the predictive and concurrent validity of CCAA scores using other data (especially qualitative data; see Winke et al, 2018), which would also enhance the interpretability of such scores. Owing to time and resource constraints, we were unable to follow fully Kane's validity framework (2013), and only focused on our assessment's content validity and psychometrics.…”

“…As the teaching and learning of young L2 Chinese learners in Hong Kong is a developing field, retaining these items will enable the CCAA to measure learners' growth across cohorts. Qualitative research is needed to understand if there is any psychological effect of presenting young children with these types of items that may be too hard for them (for an example of such qualitative research, see Winke, Lee, Ahn, Choi, Cui, & Yoon, 2018). Such findings from interviews with the children or their teachers could inform the development of such tests, and allow developers to know whether computer adaptive tests should be used to prevent low-skilled children from receiving higher-difficulty-level items.…”

English Word and Pseudoword Spellings and Phonological Awareness: Detailed Comparisons From Three L1 Writing Systems

“…Thus, the particular difficulties that learners had may have been somewhat an artifact of the particular items that were included, and thus not fully representative of overall spelling abilities. A final limitation that we would like to highlight is the challenge of using standardized assessments, such as the Elision task from the CTOPP, with L2 speaker populations (see also Greenberg et al, 2009;Pae et al, 2012;Nanda et al, 2014;Winke et al, 2018). For example, when coding spoken responses to such assessments, it can be challenging to determine accuracy while accounting for L2 accents in a consistent -and fair -way.…”