A Rationale for Criterion‐Referenced Proficiency Testing

Abstract

This article summarizes some of the technical issues that add to the complexity of language testing. It focuses in particular on the criterion‐referenced nature of the ACTFL Proficiency Guidelines—Speaking; and it proposes a criterion‐referenced interpretation of the ACTFL guidelines for reading and listening. It then demonstrates how using criterion‐referenced testing and scoring enhances the accuracy of reading and listening proficiency ratings while also providing informative feedback to learners.

Using different carrots: How incentivization affects proficiency testing outcomes

“…Zlatkin‐Troitschanskaia and Pant () pointed out that “ideally, individual students’ performance should be measured both longitudinally over the course of their studies and cross‐sectionally in comparison to that of other students from the same degree course or university (p. 254). Thus, across the K–12 and postsecondary spectrum, program‐level assessment must be viewed as “integral (rather than peripheral)” (Norris, , p. 168); otherwise, Norris () warned, assessments function only to monitor accountability and do not help educators revise their assessment practices, measure language learners’ knowledge and skills against the program's predetermined performance standards and learning expectations (Clifford, ), or support learners in tracking their progress and understand more broadly the extent to which learners’ proficiency approaches their personal and professional learning goals.…”

“…On this point, Glisan, Swender, and Surface (2013) described increasing attention to the oral proficiency levels of language instructors as part of a "renewed national focus on teacher quality and effectiveness [that] has resulted in more rigorous standards that describe the knowledge and skills required of professional standards" (p. 264). Finally, as mentioned earlier, proficiency-based assessments can be used to determine or rethink specific benchmarks for language programs across the four skills and three communicative modes, as noted by Goertler et al (2016) and Clifford (2016).…”

Understanding Intermediate-Level Speakers’ Strengths and Weaknesses: An Examination of OPIc Tests From Korean Learners of English

“…While a number of studies have looked at the validity of the OPI and the use of its scale in oral proficiency testing (Dandonoli & Henning, ; Halleck, ; Surface & Dierdorff, ; Thompson, , ), little empirical research has specifically sought to document examinees’ strengths and weaknesses at each sublevel within a major level, primarily because the single holistic rating of the speech sample as a whole results in a lack of transparency about exactly what such a rating means and on which dimensions a test taker showed strength or weakness. Thus, while the small percentage of instructors who have received formal OPI training can intuit the reason their students may have received a particular score, the large number of instructors who have less familiarity with the scale may: fail to understand the conjunctive nature of a proficiency rating (Clifford, ), overestimate their own students’ abilities (Levine & Haus, ), or confound the performance of rehearsed material with proficiency (Cox, Bown, & Burdis, ). …”

“…While learners may progress in a linear way on each of these characteristics, conjoint mastery of multiple linear characteristics is necessary for movement through one major level and into the next. Obtaining a rating at the next higher level only occurs through sustained performance of the lower levels (ACTFL, ; Clifford, ). Before a rating can be awarded, the speaker must demonstrate a sustained level or “floor” of performance across tasks, text type, content, and accuracy (see Table ) as well as a breakdown level or “ceiling” in which the examinee can no longer sustain performance in one or more of the four domains (ACTFL, ).…”

“…As shown in Table , the fundamental difference between speech that is rated at any one of the three Intermediate sublevels (Low, Mid, High) lies in the quality and quantity of the examinee's language when engaged in at‐level tasks (Clifford, ). The Low sublevel is indicative of a speaker who just barely demonstrates competence when performing the tasks for the major level.…”