Item-Score Reliability as a Selection Tool in Test Construction

Abstract

This study investigates the usefulness of item-score reliability as a criterion for item selection in test construction. Methods MS, λ6, and CA were investigated as item-assessment methods in item selection and compared to the corrected item-total correlation, which was used as a benchmark. An ideal ordering to add items to the test (bottom-up procedure) or omit items from the test (top-down procedure) was defined based on the population test-score reliability. The orderings the four item-assessment methods produced in samples were compared to the ideal ordering, and the degree of resemblance was expressed by means of Kendall's τ. To investigate the concordance of the orderings across 1,000 replicated samples, Kendall's W was computed for each item-assessment method. The results showed that for both the bottom-up and the top-down procedures, item-assessment method CA and the corrected item-total correlation most closely resembled the ideal ordering. Generally, all item assessment methods resembled the ideal ordering better, and concordance of the orderings was greater, for larger sample sizes, and greater variance of the item discrimination parameters.

Are the Teachers and Students Satisfied: Sustainable Development Mode of Entrepreneurship Education in Chinese Universities?

“…The method of reliability test was to calculate the corrected item–total correlation (CITC) of each measurement item ( Zijlmans et al, 2019 ). At the same time, the Cronbach’s alpha coefficient, which was the most widely used method for estimating internal consistency reliability ( Trizano-Hermosilla and Alvarado, 2016 ), was calculated.…”

Development and initial psychometric assessment of the race-related attitudes and multiculturalism scale in Australia

“…One important aspect in instrument development is achieving a good balance between a small number of items (lengthy questionnaires can induce lower response rates [60]) and adequate reliability. A recent study by Zijlmans, Tijmstra [61] showed that the CITC [62] performed better than other methods at identifying which items can be removed while maximizing reliability. Therefore, items with the lowest CITC should be the first to be considered for removal.…”

Exploring Factors Link to Teachers’ Competencies in Entrepreneurship Education

“…A reliability test was conducted by calculating the CITC of each measurement item. If the value was less than 0.5 (Zijlmans et al, 2019), the index was deleted. Meanwhile, Cronbach's alpha coefficient was calculated.…”

“…It is usually evaluated between 0 and 1. If the alpha coefficient is ≥0.7 (Zijlmans et al, 2019), the reliability of the indexes is acceptable. The CITC minimum value of entrepreneurial education teachers' competency scale was 0.692, Cronbach's alpha was 0.952, and the Cronbach's alpha coefficient value after deleting all the observed variables in the table was smaller than the original scale's alpha value, indicating that the scale had high reliability.…”