Linking English‐language Test Scores Onto the Common European Framework of Reference: An Application of Standard‐setting Methodology

Abstract

The Common European Framework of Reference (CEFR) describes language proficiency in reading, writing, speaking, and listening on a 6-level scale. In this study, English-language experts from across Europe linked CEFR levels to scores on three tests: the TOEFL ® iBT test, the TOEIC ® assessment, and the TOEIC Bridge™ test. Standard-setting methodology (a modified Angoff approach and a modified examinee paper selection approach) was used to construct the linkages. Linkages were established for TOEFL iBT at levels B1, B2, and C1.Linkages were established for TOEIC at levels A1 through C1, with the exception of Reading at the C1 level. The TOEIC Bridge test was linked to its three targeted levels of the CEFR. The report details the methods, procedures, and results of the study. ETS administers the TOEFL program under the general direction of a policy board that was established by, and is affiliated with, the sponsoring organizations. Members of the TOEFL Board (previously the Policy Council) represent the College Board, the GRE Board, and such institutions and agencies as graduate schools of business, two-year colleges, and nonprofit educational exchange agencies.Since its inception in 1963, the TOEFL has evolved from a paper-based test to a computer-based test and, in 2005, to an Internet-based test, TOEFL iBT. One constant throughout this evolution has been a continuing program of research related to the TOEFL test. From 1977 to 2005, nearly 100 research and technical reports on the early versions of TOEFL were published. In 1997, a monograph series that laid the groundwork for the development of TOEFL iBT was launched. With the release of TOEFL iBT, a TOEFL iBT report series has been introduced.Currently this research is carried out in consultation with the TOEFL Committee of Examiners. Its members include representatives of the TOEFL Board and distinguished English as a second language specialists from the academic community. The Committee advises the TOEFL program about research needs and, through the research subcommittee, solicits, reviews, and approves proposals for funding and reports for publication. Members of the Committee of Examiners serve four-year terms at the invitation of the Board; the chair of the committee serves on the Board. Current (2007Current ( -2008 members of the TOEFL Committee of Examiners are:

Developing and Validating Band Levels and Descriptors for Reporting Overall Examinee Performance

“…During a standard setting study, expert panelists are asked to make judgments about the performance of examinees with varied ability (as indicated by the CEFR performance descriptors) on test items. The outcome of such studies is a minimum test score that an examinee needs to achieve to be classified as minimally able to perform language tasks described at each CEFR level (see, e.g., Papageorgiou, 2010c;Tannenbaum & Wylie, 2008). Although the standard setting approach is largely subjective, empirical test performance data are also used to help panelists make their judgments and validate the cut scores.…”

“…Language test providers typically refer to the six main proficiency levels (A1-C2) of the CEFR to help score users interpret the meaning of test scores (Papageorgiou, 2010b;Tannenbaum & Wylie, 2008;Taylor & Jones, 2006). It has been argued that this use of the CEFR levels can be helpful in assisting users with score interpretation.…”

Adaptation of the CEFR to remedial English language education in Japan

“…The proficiency level of the target learners for the course is estimated to be somewhere between A2 and B1. This estimate is based on Tannenbaum and Wylie (2008). They show that the cut-scores of TOEFL and TOEIC 3 correspond to levels A1 through C1 of the CEFR.…”

Test-taker perception of and test performance on computer-delivered speaking tests: the mediational role of test-taking motivation

“…All participants had received at least seven years of high school and university-level English instruction at the time of the study, and the majority of the participants were required to take two English courses designed to improve their receptive and productive skills. Based on their TOEIC speaking test scores (M = 134.6; SD = 19.0), the speaking proficiency levels of most participants were B1 (72.2%) and A2 (16.7%), according to the Common European Framework of Reference for Languages (Tannenbaum & Wylie, 2008). Their TOEIC LR test scores (M = 767.2; SD = 115.1) showed that their receptive skills mainly belonged to B1 (51.8%) and B2 (37.0%) levels.…”