Investigating test-takers’ strategy use in task-based L2 pragmatic speaking assessment

Abstract

This study explicates cognitive validity of task-based L2 pragmatic speaking assessment by examining reported strategy use of test takers at varying performance levels across different task types. Thirty university-level ESL learners completed four pragmatic speaking tasks that differ in the formality of pragmatic actions. Two trained raters scored the task-based pragmatic performances using analytical rating criteria and displayed a satisfactory level of consistency and accuracy in scoring the performances. The test-takers’ retrospective reports were transcribed and analyzed to develop a valid coding scheme that consists of cognitive, metacognitive, and pragmatic strategies. An association between the test-takers’ pragmatic performances scored by the trained raters and their reported strategy use was examined. The higher-ability test takers utilized diverse strategies more frequently, ranging from varied pragmatic strategies to strategies specifically related to managing task demands, compared to the lower-ability test takers. Further, the test takers utilized distinct types of strategies appropriate to handling unique pragmatic task situations and complexities involved in each pragmatic assessment task. These findings explain how the test takers cognitively interacted with the assessment tasks and what strategies potentially led to successful pragmatic performances. The implications of examining pragmatic strategy use were discussed in terms of advancing practices of teaching and assessing L2 pragmatics.

Interlanguage Pragmatic Learning Strategies (IPLS) as Predictors of L2 Speech Act Knowledge : A Case of Iranian EFL Learners

“…Results of this study are consistent with the results of some earlier studies that have reported a significant association between L2 pragmatic performance across various speech acts and the use of specifically tailored strategies for acquiring pragmatic knowledge (e.g., Cohen, 2005Cohen, , 2010Cohen & Sykes, 2013;Li, 2013;Youn & Bi, 2019). Youn and Bi (2019), for example, reported a positive relationship between cognitive, metacognitive, and specific pragmatic strategies and L2 learners' pragmatic performance. They also found that higher ability test-takers used a larger range of both specific pragmatic learning strategies and general cognitive and metacognitive task-dependent strategies significantly more than lower ability test-takers.…”

“…Recent research has empirically documented the notion that the use of IPLS is not monolithic, but rather dynamic, interactive, and reciprocal in that learners may simultaneously capitalize on a cognitive, social, affective or metacognitive function from one moment to the next, which is highly contingent upon the nature of the interaction (Cohen & Wang, 2018;Malmir & Derakhshan, 2020;Tajeddin & Malmir, 2015;Youn & Bi, 2019). For instance, Cohen and Wang ( 2018) conducted a study on three ESL and three EFL Chinese-speaking university students who were required to individually perform an English vocabulary task to unveil the extent to which the use of a strategy involves more than one function.…”

Cognitive Processes of ESL Learners in Pragmatic Role-Play Tasks in Academic Settings

“…Regarding cognitive processes in specific situations, in the past decade, the pragmatic competence of L2 learners in EAP context has begun to draw attention among some strategy researchers (e.g., Cohen and Sykes, 2013 ; Youn and Bi, 2019 ) due to the increasing numbers of international students pursuing a University degree in English-speaking countries. The research reported that many international students may not be adequately prepared for cultural, societal, and interpersonal communications.…”

“…In their study on the effectiveness of strategy-based instruction, the results suggested that familiarizing learners with strategies would make a difference in their pragmatic performance. The study by Youn and Bi ( 2019 ) was one of the few empirical studies that quantitatively investigated how L2 learners at varying levels of pragmatic performance used metacognitive, cognitive, and pragmatic strategies to complete a range of pragmatic tasks in an academic setting.…”

Considerations in assessing pragmatic appropriateness in spoken language

“…Fortunately, researchers are picking up on the challenge made in an early effort to categorize pragmatics strategies (Cohen, 2005). A study by Youn and Bi (2019), for example, looked at the pragmatics strategies which 30 respondents reported using while engaging in four pragmatics English for academic purposes (EAP) tasks. The findings indicated that the strategies found to be used included metacognitive and cognitive strategies related to task-based performance, as well as strategies specific to the performance of TL pragmatics – namely, situation-related sociopragmatic, pragmalinguistic, and interactional strategies.…”