Exploring the validity and reliability of online assessment for conversational, narrative, and expository discourse measures in school-aged children

Abstract

The COVID-19 pandemic has created novel challenges in the assessment of children's speech and language. Collecting valid data is crucial for researchers and clinicians, yet the evidence on how data collection procedures can validly be adapted to an online format is sparse. The urgent need for online assessments has highlighted possible the barriers such as testing reliability and validity that clinicians face during implementation. The present study describes the adapted procedures for on-line assessments and compares the outcomes for monolingual and bilingual children of online and in-person testing using conversational, narrative and expository discourse samples and a standardized vocabulary test. A sample of 127 (103 in-person, 24 online) English monolinguals and 78 (53 in-person, 25 online) simultaneous French-English bilinguals aged 7–12 years were studied. Discourse samples were analyzed for productivity, proficiency, and syntactic complexity. MANOVAs were used to compare on-line and in-person testing contexts and age in two monolingual and bilingual school-age children. No differences across testing contexts were found for receptive vocabulary or narrative discourse. However, some modality differences were found for conversational and expository. The results from the study contribute to understanding how clinical assessment can be adapted for online format in school-aged children.

Tele-Assessment of Oral Personal Narratives in Arabic- and Hebrew-Speaking Children Using the Global TALES Protocol

“…It appears that Arabic-speaking children may have fewer opportunities to develop telecommunication skills, which may affect the length of narratives they produce remotely. This view may correspond to Burchell et al's [1] suggestion that school-aged children produce longer samples in conversation during FTF compared to TA due to disruptions in the conversational flow caused by the quality of the internet connection at times. However, it is notable that, in the current study, we did not collect any data on the accessibility of participants to telecommunication technologies or the support provided to use telecommunication.…”

“…Does the assessment mode (TA vs. FTF) impact the chosen topics of children's personal narratives, and, if so, can performance variations be attributed to a language/culture effect? Based on previous literature indicating the validity of TA of language skills [1,3], this study predicted a similar performance between TA and FTF assessment, regardless of the children's language/culture. In addition, based on a previous study comparing Arabic-and Hebrew-speaking children in Israel [34], it was predicted that the Hebrewspeaking children would produce longer personal narratives than the Arabic-speaking children, regardless of the assessment mode.…”

Feasibility of Using the Global Tales Protocol to Elicit Personal Narratives in 10-year-Old Children in Ireland

“…One of the potential reasons for the lower number of utterances and/or the higher number of non-responses to some questions may have been due to the fact that the data were collected using the online platform Zoom. However, this hypothesis is not consistent with previous studies; for example, no differences were found between in-person and online assessment of conversational, narrative, and expository discourse measures in 205 schoolaged monolingual and bilingual children (ages 7-12) [10]. Future research should investigate the effect of elicitation mode on personal narratives, with children perhaps being less inclined to share (personal) information in an online environment.…”

“…Research with children with language disorders has focused mainly on fictional narratives and story retell tasks [8]. However, personal narratives (recounts of personally experienced events) are also important because they are a naturally occurring and spontaneous form of discourse [9, 10]. In fact, most conversations between children aged five to eleven involve personal event narratives where children share their experiences with others [10].…”

“…However, personal narratives (recounts of personally experienced events) are also important because they are a naturally occurring and spontaneous form of discourse [9, 10]. In fact, most conversations between children aged five to eleven involve personal event narratives where children share their experiences with others [10]. Personal narratives are an integral part of our lives because, through sharing our stories with others, we make sense of our daily experiences [11].…”