The influence of interlocutor proficiency in a paired oral assessment

Abstract

The use of pair work in speaking assessment may encourage collaboration in the classroom and has other advantages (Saville & Hargreaves, 1999; Taylor, 2000) but from a measurement perspective, the paired oral format may be problematic because a partner may unfairly influence an examinee's performance or otherwise bias scores. In this study, the influence of interlocutor proficiency on speaking performance was examined in a group of 20 first-year students at a Chinese university. The students were divided into groups of relatively high and low English proficiency and tested once with a partner of similar proficiency and once with a partner of higher or lower proficiency. Interlocutor proficiency level had no observable effect on Rasch analysis ability measures, but lower-level examinees produced more language (words) when working with a higher-level partner. The majority of dyads produced collaborative interactions (Galaczi, 2008), unless an examinee was paired with a much lower-level partner, in which case the interaction tended to be asymmetric. Overall, these data suggest that proficiency differences among examinees need not preclude use of the paired oral test format.

Data elicitation through language testing: Challenges of test design

“…Davis (2009) attributes the complexity to the interaction of different factors and their influence on a final score. It is far beyond the scope of the paper to attempt to reiterate all the relevant factors, therefore, the major challenges perceived in the particular testing situation will be in focus.…”

“…The examinees could choose a partner, but it was based on availability rather than personal preference;however,thepotential existence of some interpersonal relationships cannot be excluded. Regarding a variety of proficiency levels, it was considered in the light of the study by Davis (2009) in which he investigates the effects of the proficiency level of an examinee's partner in a paired oral test. Davis concludes that the level of proficiency has little influence on scores, but in some cases the pairing type appears to influence language quantity or interaction characteristics (ibid.).…”

Las tareas de comunicación oral en la evaluación del inglés: el uso de conectores

“…Así se ha podido constatar que en las situaciones en las que un alumno tiene un mayor nivel lingüístico que su compañero, se produce una mayor producción oral en general (Norton, 2005), aumento que también se constata en el alumno de nivel más bajo (Davis, 2009). En otros casos, el objetivo de la investigación no ha sido la cantidad de producción oral sino la calidad de dicha producción.…”

Implementing the teaching/learning of oral skills in secondary education in Aragón: Gauging teachers’ attitudes, beliefs and expectations

“…Half of the teachers also considered it important that students know the vocabulary that is needed (58.7%) and that they feel they are not being assessed (46.0%) -the stress involved may influence learners' performance negatively. Moreover, they think that there are some factors which do not contribute to making students willing to practice productive oral skills: correcting their mistakes, as this will possibly entail their interruption, marking their performance, or all those aspects which make them perceive the task as less meaningful (Arnold, 2011;Davis, 2009). …”