Learner acquaintanceship and oral proficiency test pair-task performance

Abstract

This paper explores the effect on pair-task performance of test-takers’ familiarity with their partner. O’Sullivan (2000a), like McNamara (1996), sees test performance as being affected by a number of factors, related to the test-taker, the interlocutor and the task. Evidence from the psychology literature, and anecdotal evidence from language learners and teachers, suggests that familiarity with one’s partner in an interactive task might positively affect performance, although in the only published study to date, Porter (1991a) found no evidence to support this hypothesis. A group of 32 Japanese learners performed a series of three tasks (personal information exchange, narrative, and decision making), once with a friend, and again with a person who was not known to them. All performances were video-recorded, awarded scores by trained raters and transcribed for analysis. Results suggest evidence of an `acquaintanceship’ effect, with subjects achieving higher scores when working with a friend. However, analysis of the language suggests that there is no effect on linguistic complexity, and that there is a sex-ofinterlocutor by acquaintanceship interaction effect for accuracy. These results appear to support findings in the literature on second language acquisition, which suggest that learners vary their language when interacting with familiar or unfamiliar speakers (Plough and Gass, 1993; Tarone and Liu, 1995), although they contradict the findings of Porter (1991a). Despite its limitations, the results of this study confirm that any test format that employs tasks requiring interaction between individuals is in need of urgent and extensive study.

New and not so new methods for assessing oral communication

“…With regard to the effects of interlocutor traits, research suggests that the test takers' familiarity with other test takers (O'Sullivan, 2002), as well as personality (level of extraversion), English proficiency level, and the number of participants, may influence test takers' performance in group discussion 11 tasks, as shown in a study on Japanese secondary school students conducted by Nakatsuhara (2011). On the other hand, Ockey, Koyama, and Setoguchi (2013) investigated the effect of interlocutor familiarity on test takers' performance in a Japanese university.…”

Interpretations, intended uses and designs in task-based language assessment

“…Where such interpretations are prioritized in test use, the inferential processes upon which they are based will also need to be clearly understood and carefully operationalized. Of course, in developing such tests, it will probably be important to consider whether systematic sources of variability are being introduced through features of the tasks, or other features of the assessment setting (e.g., interlocutors; see O'Sullivan, 2002), and whether the performances elicited on test tasks provide an adequate and accurate basis for making trustworthy interpretations.…”

Untitled

“…The last group of studies on group oral testing deals with the validity issue, examining the group oral format from the perspectives of interlocutor characteristics and test discourse (Berry, 2004;Ockey, 2009;O'Sullivan, 2002;Nakatsuhara, 2011Nakatsuhara, , 2013. They argue that the examinees' personality types affect their performance to the degree that is not negligible.…”

“…They also point out that the test format leads to positive washback due to their interactive nature of oral communication while helping deal with logistic issues in test administration. Another group explored the validity aspects of the group oral format as a method to assess L2 oral proficiency (Bonk & Ockey, 2003;Fulcher, 1996;Leaper & Riazi, 2014;Shohamy, Reves, & Bejarano, 1986;O'Sullivan, 2002;Park, 2008;Van Moere, 2006). Fulcher (1996) argued that the group discussion test assessed the same language skills as other test tasks such as the picture-based discussion and text-based discussion.…”