Young learners’ response processes when taking computerized tasks for speaking assessment

Abstract

We investigated how young language learners process their responses on and perceive a computer-mediated, timed speaking test. Twenty 8-, 9-, and 10-year-old non-native English-speaking children (NNSs) and eight same-aged, native English-speaking children (NSs) completed seven computerized sample TOEFL® Primary™ speaking test tasks. We investigated the children’s attentional foci on different test components (e.g., prompts, pictures, and a countdown timer) by means of their eye movements. We associated the children’s eye-movement indices (visit counts and fixation durations) with spoken performance. The children provided qualitative data (interviews; picture-drawings) on their test experiences as well. Results indicated a clear contrast between NNSs and NSs in terms of speech production (large score differences) as expected. More interestingly, the groups’ eye-movement patterns differed. NNSs tended to fixate longer on and looked more frequently at the countdown timer than their NS peers, who were more likely to look at content features, that is, onscreen pictures meant to help with building up speech. Specifically, the NNSs’ fixations on timers were likely to co-occur with hesitation phenomena (e.g., hemming; pausing; silence). We discuss (a) the potential effects of test-specific features on children’s performance and (b) child-appropriate test accommodations and practices.

Motivational Factors in Computer-administered Integrated Skills Tasks: A Study of Young Learners

“…The less favourable motivational dispositions might also be due to the computer-mediated mode of speaking test delivery. As in Lee and Winke's (2018) study, our participants found it somewhat unusual and stressful to speak to a computer. This was also expressed in one of the participants' comments: "It was strange to talk to a machine and I felt I did not do so well."…”

“…Unlike in face-to-face communication, test-takers do not only need to make sure that they do not speak longer than required but that they fill the time with speech and start talking at the right time. Specifically with reference to young language learners, Lee and Winke (2018) found that the absence of a real audience and lack of feedback in online speaking tasks put additional pressure on the young learners and can induce predominantly negative emotional reactions to speaking tasks.…”

Assessing Speaking Proficiency: A Narrative Review of Speaking Assessment Research Within the Argument-Based Validation Framework

“…Our analysis results indicate that despite the quite frequent use of traditional qualitative methods such as interviews and focus groups, new qualitative methods that are supported by technology (e.g., eye-tracking) have only recently been utilized by speaking assessment researchers. For example, a recent study by Lee and Winke (2018) demonstrated the use of eye-tracking in speaking assessment through examining test-takers' cognitive processes when responding to computerbased speaking assessment tasks. Eye-tracking is advantageous in the sense that as opposed to traditional qualitative methods such as introspective think-aloud protocols, it causes minimal interference of the test taking process.…”

Evaluating subscore uses across multiple levels: A case of reading and listening subscores for young EFL learners

“…Despite the potential instructional value of subscores, however, few previous investigations of subscores addressed language tests specifically designed for young learners. Research on assessments of young learners' language proficiency has become quite active in the recent years (Wolf & Butler, 2017), and as a standardized assessment of young EFL learners' English proficiency, the TOEFL Primary tests have been studied in terms of their theoretical and empirical properties (Cho et al, 2016;Hsieh et al, 2018;Lee & Winke, 2018;Zu, Moulder, & Morgan, 2017). In addition, the added-value of subscores from the TOEFL Primary Reading and Listening sections have been examined using Haberman's (2008) method (Papageorgiou & Choi, 2018), based on data collected from several operational administrations mostly in South American and Asian countries.…”