Stakeholders' Beliefs About the <i><scp>TOEFL iBT</scp></i><sup>®</sup> Test as a Measure of Academic Language Ability

Malone, Margaret E.; Montee, Megan

doi:10.1002/ets2.12039

Cited by 17 publications

(19 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The survey consisted of a combination of multiple‐choice and open‐ended questions based in part on earlier studies investigating the use of test scores (e.g., Hyatt & Brooks, ; Malone & Montee, ; O'Loughlin, ). There were several differences between the Japanese and other two survey versions because data collection plans for Mexico and Indonesia were finalized several months after those for Japan, providing for an opportunity to revisit and tweak survey content.…”

Section: Methodsmentioning

confidence: 99%

“…There has been limited research to date examining institutions' actual uses of TOEFL ITP scores and perceptions of this assessment (though case studies of particular uses have been reported, e.g., Choi & Papageorgiou, ; Minton & Nishikawa, ). On the other hand, there is an accumulating body of research on the use of TOEFL iBT and IELTS scores (e.g., Ginther & Elder, ; Hyatt & Brooks, ; Malone & Montee, ; O'Loughlin, ; Stricker & Attali, ; Stricker & Wilder, ). Relative to assessments like TOEFL ITP, these tests have more involved development, administrative, and scoring procedures and are typically available at a higher price point.…”

Section: Investigating Applications and Perceptions Of Toefl Itp Scoresmentioning

confidence: 99%

See 1 more Smart Citation

Examining the Applications and Opinions of the TOEFL ITP® Assessment Series Test Scores in Three Countries

Golubovich

Tolentino

Papageorgiou

2018

ETS Research Report Series

View full text Add to dashboard Cite

In this study, 249 users of the TOEFL ITP® assessment series (e.g., admissions officers, English‐language teachers, academic staff) in Japan, Mexico, and Indonesia were surveyed about their uses and opinions of TOEFL ITP scores, followed by in‐depth interviews with 21 of these users. Overall, the most common use of the test was as an exit requirement from English‐language programs to demonstrate proficiency in English listening and reading. The majority of participants saw TOEFL ITP scores as very useful indicators of students' English‐language proficiency. Interviews helped clarify the user needs met by this assessment and how test scores were actually applied. Study participants indicated that they need a relatively inexpensive and practical English‐language assessment that also provides them with enough information to make decisions about test takers' proficiency in all relevant skill areas. Some of the ways interviewees talked about using TOEFL ITP scores were consistent with the recommendations of Educational Testing Service (ETS), while other uses (e.g., for workplace applications) were more questionable, as they might imply potentially higher stakes than were intended for this test. The results of the study highlight areas where TOEFL ITP users might need additional informational support with regard to score interpretation and use.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Investigating Applications and Perceptions Of Toefl Itp Scoresmentioning

confidence: 99%

Examining the Applications and Opinions of the TOEFL ITP® Assessment Series Test Scores in Three Countries

Golubovich

Tolentino

Papageorgiou

2018

ETS Research Report Series

View full text Add to dashboard Cite

show abstract

“…In 2005, the Educational Testing Service (ETS) first released the Test of English as a Foreign Language Internet-based Test (TOEFL iBT), which is now the most common test of English ability used in US college admissions [6]. Evidence for correlation with scores from its paper-based predecessor, which had no speaking section, and SPEAK Test scores was found in 1994 at Iowa State University; the study's author noted "Indo-European language speakers… obtained much higher SPEAK scores" than Chinese or Korean students [7].…”

Section: An English Test For Admissions and Screeningmentioning

confidence: 99%

Forecasting SPEAK test score from TOEFL score: a Bayesian model for screening international teaching assistants

Gurlitz

2015

2015 Systems and Information Engineering Design Symposium

View full text Add to dashboard Cite

Teaching assistantships provide funding to graduate students and support to teaching faculty. At many universities in the United States, students whose first language is not English can only teach if they pass an oral English assessment or complete English courses. Graduate programs face a risk that a prospective international teaching assistant (ITA) may fail the English test, potentially causing financial and logistical consequences. To develop rational decision procedures, which integrate the probability of failure on the test with the consequences and tradeoffs of ITA funding decisions, a probabilistic forecasting model is needed.At the University of Virginia (UVa), ITAs must pass the Speaking Proficiency English Assessment Kit (SPEAK) Test before the semester begins in August. In this study, scores from the Test of English as a Foreign Language Internet-based Test (TOEFL iBT), which are available at the admission time, are used to produce a probability of failing the SPEAK Test at a given threshold. A Bayesian forecasting model is described, estimating the prior probabilities from a sample of SPEAK Test scores of 803 prospective ITAs at UVa between 2006 and 2013, and using the TOEFL iBT scores from 318 students to update the forecast probabilities. Additional discrete predictors, such as the student's gender and school, are considered, but only the native language is found to reduce uncertainty in the forecast. Overall, this forecasting model demonstrates and explains a useful statistical association between the SPEAK Test scores and the TOEFL iBT scores, used widely in university admissions.

show abstract

“…Arguably, "the purpose of the essay task, whether for diagnosis, development, or promotion, is significant in deciding which scale is chosen" (Brossell 1986, p. 2). As rubrics should be conceived and designed with the purpose of assessment of any given type of written task (Crusan 2015;Fulcher 2010;Knoch 2009;Malone and Montee 2014;Weigle 2002), the development and validation of rating scales are very challenging issues.…”

Section: Introductionmentioning

confidence: 99%

Examining consistency among different rubrics for assessing writing

Shabani

Panahi

2020

Lang Test Asia

View full text Add to dashboard Cite

The literature on using scoring rubrics in writing assessment denotes the significance of rubrics as practical and useful means to assess the quality of writing tasks. This study tries to investigate the agreement among rubrics endorsed and used for assessing the essay writing tasks by the internationally recognized tests of English language proficiency. To carry out this study, two hundred essays (task 2) from the academic IELTS test were randomly selected from about 800 essays from an official IELTS center, a representative of IDP Australia, which was taken between 2015 and 2016. The test takers were 19 to 42 years of age, 120 of them were female and 80 were males. Three raters were provided with four sets of rubrics used for scoring the essay writing task of tests developed by Educational Testing Service (ETS) and Cambridge English Language Assessment (i.e., Independent TOELF iBT, GRE, CPE, and CAE) to score the essays which had been previously scored officially by a certified IELTS examiner. The data analysis through correlation and factor analysis showed a general agreement among raters and scores; however, some deviant scorings were spotted by two of the raters. Follow-up interviews and a questionnaire survey revealed that the source of score deviations could be related to the raters’ interests and (un)familiarity with certain exams and their corresponding rubrics. Specifically, the results indicated that despite the significance which can be attached to rubrics in writing assessment, raters themselves can exceed them in terms of impact on scores.

show abstract

Stakeholders' Beliefs About the TOEFL iBT^® Test as a Measure of Academic Language Ability

Cited by 17 publications

References 40 publications

Examining the Applications and Opinions of the TOEFL ITP® Assessment Series Test Scores in Three Countries

Examining the Applications and Opinions of the TOEFL ITP® Assessment Series Test Scores in Three Countries

Forecasting SPEAK test score from TOEFL score: a Bayesian model for screening international teaching assistants

Examining consistency among different rubrics for assessing writing

Contact Info

Product

Resources

About

Stakeholders' Beliefs About the TOEFL iBT® Test as a Measure of Academic Language Ability

Cited by 17 publications

References 40 publications

Examining the Applications and Opinions of the TOEFL ITP® Assessment Series Test Scores in Three Countries

Examining the Applications and Opinions of the TOEFL ITP® Assessment Series Test Scores in Three Countries

Forecasting SPEAK test score from TOEFL score: a Bayesian model for screening international teaching assistants

Examining consistency among different rubrics for assessing writing

Contact Info

Product

Resources

About

Stakeholders' Beliefs About the TOEFL iBT^® Test as a Measure of Academic Language Ability