Playing the Recording Once or Twice: Effects on Listening Test Performances

Ruhm, Richard; Leitner-Jones, Claire; Kulmhofer, Andrea; Kiefer, Thomas; Mlakar, Heike; Itzlinger-Bruneforth, Ursula

doi:10.1080/10904018.2015.1104252

Cited by 9 publications

(5 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While there seems to be a general tendency for item difficulty to decrease if test-takers are given the possibility to listen more than once (Field, 2015; Holzknecht, 2019; Ruhm et al, 2016), the effect of replay in self-paced listening on listening scores remains unclear. Roussel (2011) showed in a small-scale study ( n = 29) that learners of L2 German aged between 14 and 16 years scored higher in a self-paced condition in contrast to administrator-paced single or double play conditions.…”

Section: Self-paced Listeningmentioning

confidence: 99%

Investigating the impact of self-pacing on the L2 listening performance of young learner candidates with differing L1 literacy skills

et al. 2023

View full text Add to dashboard Cite

In online environments, listening involves being able to pause or replay the recording as needed. Previous research indicates that control over the listening input could improve the measurement accuracy of listening assessment. Self-pacing also supports the second language (L2) comprehension processes of test-takers with specific learning difficulties (SpLDs) or, more specifically, of learners with reading-related learning difficulties who might have slower processing speed and limited working memory capacity. Our study examined how L1 literacy skills influence L2 listening performance in the standard single-listening and self-paced administration mode of the listening section of the Test of English as a Foreign Language (TOEFL) Junior Standard test. In a counterbalanced design, 139 Austrian learners of English completed 15 items in a standard single-listening condition and another 15 in a self-paced condition. L1 literacy skills were assessed via a standard reading, non-word reading, word-naming, and non-word repetition test. Generalized Linear Mixed-Effects Modelling revealed that self-pacing had no statistically significant effect on listening scores nor did it boost the performance of test-takers with lower L1 literacy scores indicative of reading-related SpLDs. The results indicate that young test-takers might require training in self-pacing or that self-paced conditions may need to be carefully implemented when they are offered to candidates with SpLDs.

show abstract

Section: Self-paced Listeningmentioning

confidence: 99%

Investigating the impact of self-pacing on the L2 listening performance of young learner candidates with differing L1 literacy skills

et al. 2023

View full text Add to dashboard Cite

show abstract

“…In contrast to the findings discussed above, the majority of studies which have investigated the effects of double play in L2 listening pedagogy and assessment found that it was beneficial compared to single play in terms of (1) quantity and accuracy of recalled lexical items and propositions (Lund, 1991), (2) an increase in perceived listening performance by students (Kwon & Park, 2017), and (3) actual listening performance as indicated through test scores (Aryadoust, 2020; Berne, 1995; Chang & Read, 2006; Field, 2015; Iimura, 2007; Ruhm et al., 2016; Sakai, 2009; Sherman, 1997). Similar to the research above, however, notable limitations of these studies include small sample sizes, that is, fewer than 50 participants in each condition (Aryadoust, 2020; Berne, 1995; Chang & Read, 2006; Iimura, 2007; Ruhm et al., 2016; Sakai, 2009; Sherman, 1997) or the use of listening tasks developed without field testing and/or prior statistical item analysis (Aryadoust, 2020; Berne, 1995; Chang & Read, 2006; Kwon & Park, 2017; Lund, 1991; Sherman, 1997). In addition, none of the studies adopted a fully counter‐balanced research design where task ordering effects were considered.…”

Section: Literature Reviewmentioning

confidence: 99%

“…This is a powerful consideration given the need to maximize measurement quality within the time constraints of operational testing. On the second consideration, a common argument for single play is that in many real‐life situations people hear a listening text only once (e.g., an important announcement; a lecture), and thus listening assessments should replicate the real‐world domain to draw valid inferences of future listening performance (Buck, 2001; Ruhm et al., 2016). Finally, single play might be implemented to enhance the psychometric quality of a listening test: to make a test more challenging, or to discriminate more effectively between candidates of different listening abilities, which might be a primary consideration depending on test purpose (Ruhm et al., 2016).…”

Section: Introductionmentioning

confidence: 99%

Repeating the Listening Text: Effects on Listener Performance, Metacognitive Strategy Use, and Anxiety

Holzknecht

Harding

2023

TESOL Quarterly

View full text Add to dashboard Cite

In second language listening assessment and pedagogy, practitioners hold different views on whether to repeat a listening text in contexts where inferences about listening ability are to be drawn from task performance. To address this issue, we investigated the effects of repeating the listening text (double play) on listener performance, listening strategies, test‐taking strategies, test‐taking anxiety, and listening anxiety. Three hundred six Austrian secondary school students responded to four listening tasks drawn from the Austrian Matura exam and completed questionnaires measuring strategic behavior and anxiety in a counter‐balanced research design. Data were analyzed using many‐facet Rasch measurement (MFRM), factor analysis, and inferential statistics. Findings confirmed that double play led to higher levels of listener performance across two task types (multiple‐choice items and note completion), however, scores were higher in the single play condition compared to the first play of double play. Students also reported lower levels of anxiety, and the use of more listening strategies and fewer test‐taking strategies in double play compared to single play with small effect sizes. We discuss the importance of balancing an empirically derived understanding of the effects of repeating the listening text with considerations of the purpose of an assessment, and the characteristics of the target language use domain.

show abstract

“…These pieces of information elicit particular focus. It is important-especially for items that focus on retrieving explicitly stated information-to have in mind that students are not able to go back to information if they missed it and that students listen to the text only once (Ruhm et al, 2016). Thus, only central information is asked in these kinds of items.…”

Section: Austriamentioning

confidence: 99%

Listening comprehension tests in Germany and Austria―Research report and critical review

Weirich¹,

Bachinger²,

Trendtel³

et al. 2019

L1 Educational Studies in Language and Literature

View full text Add to dashboard Cite

This article focuses on listening comprehension in large-scale assessments in Austria and Germany. L1 listening comprehension tests for elementary school students are part of the national educational assessment in both countries. The aim of this paper is to give an overview of the similarities and differences of these listening comprehension tests in Austria and Germany. Thus, we describe the educational policies and the underlying construct for the assessments. We will show that although both assessments are based on the same theoretical frameworks, test developers and policy makers made some different decisions for example about test procedures, statistical models and performance level descriptions (PLDs). Moreover, we illustrate that the choice of an appropriate statistical model is driven by empirical as well as didactical needs, which are difficult to reconcile with each other. These insights are illustrated by exemplary tasks and empirical examples. We use data from large-scale assessments (LSAs) from both countries, the BIST-Ü pilot study conducted by the BIFIE in Austria (N = 2,798) and the VERA-study conducted by the IQB in Germany (N = 3,107). We then draw conclusions primarily focusing on improving future tests and possible joint studies.

show abstract

Playing the Recording Once or Twice: Effects on Listening Test Performances

Cited by 9 publications

References 15 publications

Investigating the impact of self-pacing on the L2 listening performance of young learner candidates with differing L1 literacy skills

Investigating the impact of self-pacing on the L2 listening performance of young learner candidates with differing L1 literacy skills

Repeating the Listening Text: Effects on Listener Performance, Metacognitive Strategy Use, and Anxiety

Listening comprehension tests in Germany and Austria―Research report and critical review

Contact Info

Product

Resources

About