Simulating conversations in oral-proficiency assessment: a conversation                 analysis of role plays and non-scripted interviews in language exams

This study explores the nature of co-constructed interaction in group oral tests by examining whether a test-taker's own and his or her group members' extraversion levels and oral proficiency levels have different influences on conversational styles between two group sizes: groups of three and groups of four. Data were collected from 269 Japanese upper-secondary school students, who took group oral tests either in groups of three or four. All sessions were video-taped and transcribed following Conversation Analysis (CA) conventions. The data were quantitatively analysed in terms of goal-orientation, interactional contingency and quantitative dominance. Then, CA methodology was used to interpret and elaborate the statistical results. The findings have implications for our understanding of the group oral test construct and for appropriate choices of group size in group oral testing.

Section: Methods Of Data Analysismentioning

confidence: 99%

Effects of test-taker characteristics and the number of participants in group oral tests

Nakatsuhara

2011

“…They do not explicitly rate test takers' interactional language use and actually encourage extensive monologic production to obtain a ratable sample of candidate performance. While the OPI often contains a role play, which does allow test takers to deploy some of the interactional abilities that they need in real-world interactions (Okada, 2009), it would need to be investigated further (Kormos, 1999;van Lier, 1989) if it were to be used as a tool for assessing interactional abilities.…”

Section: Measuring a Broader Construct Of Pragmatic Abilitymentioning

confidence: 99%

Testing of second language pragmatics: Past and future

Roever

2011

149

Testing of second language pragmatic competence is an underexplored but growing area of second language assessment. Tests have focused on assessing learners' sociopragmatic and pragmalinguistic abilities but the speech act framework informing most current productive testing instruments in interlanguage pragmatics has been criticized for under-representing the construct. In particular, the assessment of learners' ability to produce extended monologic and dialogic discourse is a missing component in existing assessments. This paper reviews existing tests and argues for a discursive reorientation of pragmatics tests. Suggestions for tasks and scoring approaches to assess discursive abilities while maintaining practicality are provided, and the problematicity of native speaker benchmarking is discussed.

“…A possible solution for this dilemma is to do an assessment of sub-components on a principled basis. For instance, it is possible to have a series of tasks (e.g., a series of role plays) that assess language performance in different combinations of sociolinguistic conditions (see Raffaldini, 1988;Shohamy, 1988;Kormos, 1999). In general, the inadequate or inconsistent rating of speech samples is predicated on two main factors: 1) the lack of identification of the criteria used to rate performance within each module of communicative language ability; and 2) the relative weight attributed to the different components of communicative ability (see Section 5 below).…”

Section: Reliability: Separation Of Assessment Criteriamentioning

confidence: 99%

Revising the revised format of the ACTFL Oral Proficiency Interview

Salaberry

2000

Since the early 1980s proponents of proficiency examinations such as ACTFL (American Council on the Teaching of Foreign Languages) have been criticized for the low validity and reliability of tests such as the OPI (Oral Proficiency Interview). Despite these strong concerns, the most recent edition of the ACTFL Tester Training Manual does not reveal substantial changes from the previous manual published in . While a complete elimination of proficiency tests such as the ACTFL-OPI may be neither feasible nor necessary, some practical changes may be appropriate. In this article I describe minor structural changes of the ACTFL-OPI framework that would not eliminate the practical benefits of a proficiency test, but which address substantial concerns related to the validity and reliability of the instrument.