Effects of test-taker characteristics and the number of participants in group oral tests

Abstract

This study explores the nature of co-constructed interaction in group oral tests by examining whether a test-taker's own and his or her group members' extraversion levels and oral proficiency levels have different influences on conversational styles between two group sizes: groups of three and groups of four. Data were collected from 269 Japanese upper-secondary school students, who took group oral tests either in groups of three or four. All sessions were video-taped and transcribed following Conversation Analysis (CA) conventions. The data were quantitatively analysed in terms of goal-orientation, interactional contingency and quantitative dominance. Then, CA methodology was used to interpret and elaborate the statistical results. The findings have implications for our understanding of the group oral test construct and for appropriate choices of group size in group oral testing.

Untitled

“…A possible reason for contradictory findings between this study and the previous studies (e.g., KondoBrown, 2002;Nakatsuhara, 2011;Winke & Gass, 2013) could be the use of different raters in different assessment settings and different test items and tasks. For example, Kondo-Brown (2002) used only trained raters in assessing test takers' writing ability, and Winke and Gass (2013) used raters from Spanish, Chinese and Korean backgrounds.…”

“…Various groups of raters may differ in the judgment of learners' second language ability depending on their backgrounds and the criteria they apply (Barrett, 2001). Several studies have found differences between inexperienced and experienced raters in their scorings and their use of rating strategies (Khabbazbashi, 2017;Nakatsuhara, 2011;Winke & Gass, 2013). Khabbazbashi (2017) compared the ratings of trained and untrained raters from two various backgrounds: experienced English teachers and non-teachers.…”

“…14, No. 4, Winter 2017, 687-702 training cannot totally eliminate rater variability, it can make raters more self-consistent (e.g., Bijani, 2010;Bijani & Fahim, 2011;Davis, 2016;Elder, Barkhuizen, Knoch, & Randow, 2007;Khabbazbashi, 2017;Nakatsuhara, 2011).…”

“…The last group of studies on group oral testing deals with the validity issue, examining the group oral format from the perspectives of interlocutor characteristics and test discourse (Berry, 2004;Ockey, 2009;O'Sullivan, 2002;Nakatsuhara, 2011Nakatsuhara, , 2013. They argue that the examinees' personality types affect their performance to the degree that is not negligible.…”

New and not so new methods for assessing oral communication

“…With regard to the effects of interlocutor traits, research suggests that the test takers' familiarity with other test takers (O'Sullivan, 2002), as well as personality (level of extraversion), English proficiency level, and the number of participants, may influence test takers' performance in group discussion 11 tasks, as shown in a study on Japanese secondary school students conducted by Nakatsuhara (2011). On the other hand, Ockey, Koyama, and Setoguchi (2013) investigated the effect of interlocutor familiarity on test takers' performance in a Japanese university.…”