Digital literacy scale: Validity and reliability study with the rasch model

Abstract

The study aimed to develop a Likert-type measurement tool (Digital Literacy Scale, DLS) to determine the digital literacy levels of secondary school students. The validity and reliability of the developed measurement tool were verified using the Rasch model. The Rasch Model can estimate missing data and allow for small study groups (Rasch, Rasch, Probabilistic models for some intelligence and attainment tests, Danish Institute for Educational Research, 1960). Additionally, this model can verify the expected pattern of the measurement tool by calibrating it across participants and items. The content validity of DLS was ensured by expert opinion, and the construct validity was ensured by using the Rach model. In the content validity study, a 25-item pool was created for the draft DLS, and 5 items were removed from the draft DLS at this stage. Construct validity studies were carried out with the remaining 20 items using the classical test theory and item response theory (Rasch model). Exploratory Factor Analysis (EFA) and first-level Confirmatory Factor Analysis (CFA) were used within the scope of classical test theory. Then some Rasch assumptions such as dimensionlessness, local independence, monotonicity, and bias were tested for DLS. In the validity and reliability analysis of DLS, no items were eliminated, and the quantitative theoretical results were statistically confirmed. The analysis results showed that the minimum statistical values required for a good measurement tool were met. Accordingly, 20 valid and reliable compatible items that can be used to determine the digital literacy status of secondary school students were produced.