Evaluating interactional competence in interview and paired discussion tasks: A rater cognition study

Abstract

Studies of interaction in speaking assessment have highlighted problems regarding the unequal distribution of interaction patterns in different task types. However, little attempt has been made to include both verbal and nonverbal interaction features elicited in the tasks. Therefore, this study examined the elicitation of verbal and nonverbal interaction in different task types by investigating which types of interaction features raters noticed when rating interaction across interview and paired discussion tasks. The study analyzed the use of interaction features in 32 verbal reports from four raters who commented on interaction features that affected their judgment. The findings of the study suggest that both verbal and nonverbal communication contributed to interactional effectiveness. The study also revealed that test-takers seem to have more opportunities to demonstrate their interactional ability in paired task formats than in interview formats. Pedagogical implications are also provided. 2 of 18 | VO of L2 speaking assessment raises questions about what L2 speaking is or what constitutes L2 speaking ability.L2 speaking ability is defined as "the use of oral language to interact directly and immediately with others … with the purpose of engaging in, acquiring, transmitting, and demonstrating knowledge" (Jamieson, Eignor, Grabe, & Kunnan, 2008, p. 74). Based on this definition, Ockey and Li (2015) defined the construct of speaking based on four components: interactional competence (IC), appropriate use of phonology, appropriate and accurate use of vocabulary and grammar, and appropriate fluency.Many task types have been created to assess the construct of oral communication. One of the widely used task types is the oral proficiency interview. In an interview assessment, test-takers interact with a language tester who conducts the interview based on a predetermined protocol. In this interaction, the language tester asks questions and the test-taker gives answers. However, this interview format is not likely to elicit authentic discourse as in a conversation (Van Lier, 1989). It is also unlikely to measure some aspects of IC, like taking turns, opening and closing gambits, and developing topics with appropriate pragmatic use (Ockey & Li, 2015).To that end, the development of more authentic task types, such as paired or group assessments, where two or more test-takers engage in a task together without an examiner's involvement, is necessary. This is important because "the oversimplified view on human interactions taken by the proficiency movement can impair and even prevent the attainment of true interactional competence within a cross-cultural framework and jeopardize our chances of contributing to interactional understanding" (Kramsch, 1986, p. 367). Thus, it has been argued that IC should be explicitly incorporated into the concept of communicative competence (Kramsch, 1986;He & Young, 1998). Given that speaking tests, such as Cambridge English Qualifications, have evolved to capture interaction, a definition...

Investigation into the factor structure of a local English placement test of oral communication

“…Previous studies have discovered that novice and developing raters who did not have much rating experience had difficulties understanding and making clear distinctions between scoring categories that assessed separate sub-constructs (Barkaoui, 2011;Kim, 2011Kim, , 2015Kim, , 2019. In fact, earlier research investigating EPT OC raters' cognitive processes documented raters commented on features of fluency and grammar/vocabulary while judging interactional competence, which suggests the raters treated the three sub-constructs in practically identical ways (Vo, 2021).…”