Innovative Item Formats in Computer-Based Testing: In Pursuit of Improved Construct Representation

Sireci, Stephen G.; Zenisky, April L.

doi:10.4324/9780203874776.ch14

Cited by 44 publications

(59 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…the innovative formats may facilitate more authentic and direct measurement of knowledge, skills, and abilities (KSA) than the MC format allows (Parshall et al, 2002;Scalise & Gifford, 2008;Sireci & Zenisky, 2006). The innovative formats may expand the content coverage of a testing program through the use of multi-media (Swygert & Contreras, 2001;Vispoel, Wang, & Bleiler, 1997).…”

mentioning

confidence: 97%

See 1 more Smart Citation

Measurement Properties of Two Innovative Item Formats in a Computer-Based Test

Wan

Henly

2012

Applied Measurement in Education

View full text Add to dashboard Cite

Many innovative item formats have been proposed over the past decade, but little empirical research has been conducted on their measurement properties. This study examines the reliability, efficiency, and construct validity of two innovative item formats-the figural response (FR) and constructed response (CR) formats used in a K-12 computerized science test. The item response theory (IRT) information function and confirmatory factor analysis (CFA) were employed to address the research questions. It was found that the FR items were similar to the multiple-choice (MC) items in providing information and efficiency, whereas the CR items provided noticeably more information than the MC items but tended to provide less information per minute. The CFA suggested that the innovative formats and the MC format measure similar constructs. Innovations in computerized item formats are reviewed, and the merits as well as challenges of implementing the innovative formats are discussed.Recent years have seen rapid growth in the use of computers in educational and licensure/certification testing. Many tests are now administered on computer, and software has been developed to facilitate all aspects of test development. On paper-and-pencil tests, multiple-choice (MC) items are the dominant testing format (Scalise & Gifford, 2008). Now armed with new technologies, test users and developers have shown increased interest in employing innovative item formats.Many innovative item formats have emerged over the past decade, but little empirical research has been conducted on their measurement properties. The purpose of this study is to empirically examine the reliability, efficiency, and Correspondence should be addressed to Lei Wan,

show abstract

mentioning

confidence: 97%

“…For example, computer literacy may be a source of construct irrelevant variance, and the lack of research and guidance on writing innovative items poses significant challenges for the widespread implementation of the novel formats (Huff & Sireci, 2001;Sireci & Zenisky, 2006). Zenisky and Sireci (2002) reviewed 21 computerized innovative item types.…”

mentioning

confidence: 99%

Measurement Properties of Two Innovative Item Formats in a Computer-Based Test

Wan

Henly

2012

Applied Measurement in Education

View full text Add to dashboard Cite

show abstract

“…Accordingly, scoring is included in the framework of mode effects for the NEPS. In general, CBT enables automatic scoring for a variety of selectedresponse formats (e.g., multiple-choice) and for some simple constructed-response formats (e.g., short text answers, text highlighting, see Sireci and Zenisky 2006). For other response modes (e.g., complex essay scoring), automatic scoring is still under research (e.g., Haberman and Sinharay 2010).…”

Section: Test Scoringmentioning

confidence: 98%

11 Computer-based competence tests in the national educational panel study: The challenge of mode effects

Kröhne

Martens

2011

Z Erziehungswiss

View full text Add to dashboard Cite

Computerized competence tests promise a variety of advantages compared to paperpencil delivered tests, for instance, increased test security, more information about test takers and the test-taking process, instant scoring, and immediate feedback. Moreover, new innovative item types can be administered to broaden the test content. Three benefits should be particularly emphasized for the assessment of cognitive competencies in the German National Educational Panel Study. First, reductions of test time can be obtained through the higher measurement efficiency of adaptive tests. Second, computerized testing is expected to enhance standardization and to increase test takers' interest in completing the test. Third, Internet-based assessment offers the opportunity to deliver tests to spatially distanced test takers. However, before we can exploit these opportunities, we have to study the equivalence between different test administrations in order to maintain comparability of test scores and to ensure the validity of score interpretations. In this chapter, we shall describe a theoretical framework of mode effects and discuss various properties of test administrations. We shall relate the resulting equivalence criteria to the specific settings of the National Educational Panel Study in which (a) the usage of computerized competence tests is being prepared for upcoming assessments, and (b) tests for different grades and age groups are being designed to assess competence development over the life span.

show abstract

“…Computerized assessments bring many advantages to examinees compared to more conventional paper-based assessments. For instance, computers support the creation of new alternative item types and innovative item formats (Sireci and Zenisky 2006); items on computer-based tests can be scored immediately to offer immediate feedback to the examinees (Drasgow and Mattern 2006); and computers also allow on-demand testing (van der Linden and Glas 2010). However, the most significant benefit of computerized assessment is that it permits examiners to assess more complex performances of students via integrative test items which may include digital media to increase the types of skills, knowledge, and competencies that can be evaluated (Bartram 2006).…”

Section: Computer Adaptive Testingmentioning

confidence: 99%

An overview of differential item functioning in multistage computer adaptive testing using three-parameter logistic item response theory

Sadeghi

Khonbi

2017

Lang Test Asia

View full text Add to dashboard Cite

As perfectly summarised by Ida Lawrence, "Testing is growing by leaps and bounds across the world. There is a realization that a nation's well-being depends crucially on the educational achievement of its population. Valid tests are an essential tool to evaluate a nation's educational standing and to implement efficacious educational reforms. Because tests consume time that otherwise could be devoted to instruction, it is important to devise tests that are efficient. Doing so requires a careful balancing of the contributions of technology, psychometrics, test design, and the learning sciences. Computer adaptive multistage testing (MSCAT) fits the bill extraordinarily well; unlike other forms of adaptive testing, it can be adapted to educational surveys and student testing. Research in this area will be an evidence that the methodologies and underlying technology that surround MSCAT have reached maturity and that there is a growing acceptance by the field of this type of test design" (from the Foreword to Y. Duanli, A. A. von Davier, & L. Charles (Eds.), Computerized multistage testing: theory and application). This state-of-the-art paper aims to present an overview of differential item functioning (DIF) in MSCAT using three-parameter logistic item response theory (IRT), offering suggestions to implement it in practice with a hope to motivate testing and assessment researchers and practitioners to initiate projects in this under-practiced area by helping them to better understand some of the relevant technical concepts.

show abstract

Innovative Item Formats in Computer-Based Testing: In Pursuit of Improved Construct Representation

Cited by 44 publications

References 0 publications

Measurement Properties of Two Innovative Item Formats in a Computer-Based Test

Measurement Properties of Two Innovative Item Formats in a Computer-Based Test

11 Computer-based competence tests in the national educational panel study: The challenge of mode effects

An overview of differential item functioning in multistage computer adaptive testing using three-parameter logistic item response theory

Contact Info

Product

Resources

About