Developments and Challenges in the Use of Computer‐Based Testing for Assessing Second Language Ability

Abstract

Computer‐based testing (CBT) to assess second language ability has undergone remarkable development since Garret (1991) described its purpose as “the computerized administration of conventional tests” in The Modern Language Journal. For instance, CBT has made possible the delivery of more authentic tests than traditional paper‐and‐pencil tests. CBT has also made it possible to more reliably, practically, and almost instantaneously score essays, oral speech samples, and other types of test responses. Unfortunately, however, due to a number of unresolved problems, CBT has failed to realize its anticipated potential. CBT has limited usability because systems that ensure test and score security have yet to be developed. Computer‐adaptive testing, one of the most promising areas of CBT has not met expectations because of failure to solve problems with the statistical techniques on which it is based and the lack of resources necessary to implement it in most assessment contexts. In spite of these and other limitations, given the growing capability of CBT to deliver more authentic tests than paper‐and‐pencil, its use for assessing second language ability will undoubtedly continue to expand.

Computer‐Assisted Language Testing

“…Several major language tests have recently implemented tasks that assess the integrated skills of speaking and listening (Versant English Test); reading-listening-writing and listening/reading-speaking (TOEFL iBT); and reading-writing, listening-writing, listening-speaking, and readingspeaking (PTE Academic). Although Ockey (2009) maintains that "the future of integrated skills tests appears bright" (p. 845), the use of integrated tasks in CBTs poses certain challenges, namely the vagueness of language ability constructs being measured by such tests. This demands more research on multidimensional constructs and on inferences that can be made about test takers' language proficiency based on their scores for integrated items (Plakans, 2009b).…”

“…(See Carr, 2006, for a more detailed discussion of Web-based language testing.) Some researchers (e.g., Ockey, 2009) predict that due to rapid technological advances WBT will gain more popularity and witness further development in the near future.…”

“…Unlike the testing of unitary skills such as speaking, listening, reading, and writing, this type of assessment is believed to be more authentic due to the interactive nature of tasks that resemble what test takers may encounter in real-world situations (Jamieson, 2005;Ockey, 2009). Several major language tests have recently implemented tasks that assess the integrated skills of speaking and listening (Versant English Test); reading-listening-writing and listening/reading-speaking (TOEFL iBT); and reading-writing, listening-writing, listening-speaking, and readingspeaking (PTE Academic).…”

“…Compared to adaptive tests that are adaptive at the item level (i.e., by selecting the next item based on the test taker's performance on the current item), semi-adaptive tests are adaptive at the level of a group of items called testlets (Winke & Fei, 2008) or at the level of the whole test where test takers are given a version of the test that corresponds to their proficiency level as determined by a pretest (Ockey, 2009). It should be noted, however, that the term "semi-adaptive" is not universally accepted and, while some researchers distinguish semi-adaptive tests from purely adaptive tests (e.g., Winke, 2006;Ockey, 2009;Winke & Fei, 2008), others seem to consider such tests to be a specific type of adaptive test (e.g., Alderson, 2005;Jamieson, 2005).…”

“…Integrated skills assessment reflects the complexity of language use contexts (Chapelle, Grabe, & Berns, 2000) and is believed to enhance the authenticity of language tests through interactivity provided by integrated tasks (Ockey, 2009) that are typically performance-based (Plakans, 2009a). Integrated skills assessment, for instance, has been included in the new TOEFL iBT in order "to better align the test design to the variety of language use tasks that examinees are expected to encounter in everyday academic life" (Sawaki, Stricker, & Oranje, 2009).…”