Development, Reliability, and Validity of Open-ended Test to Measure Student’s Digital Literacy Skill

Abstract

This study aims to developing open-ended test for measuring digital literacy skills of students. There are three step of this study, it was defining the construc and formatting objectives, validity by expert review and item administration. The open-ended test was develop based on five components of digital literacy skill: information, communication, content creation, safety, and problem-solving. The open-ended test was initially piloted on four group of samples: interviews and validity by physics expert (2 Professor, 1 Doctoral), professional teachers (N=2), and college students of graduate school (N=2). Modification were made afterwards and the test was administered to a group of science students from high school in Yogyakarta province, Indonesia (N=129). Result of this study are: (1) The open-ended test instrumen proved valid and worthy of use to measure digital literacy skill; (2) Based on expert validation, the teset is valid with CVR value 1.00; (3) the average value and standard deviation of INFIT MNSQ is .83-1.21 (fit with rasch model/good item); (4) The item has a good degree of difficulty with range of difficulty level between (moderate level). The student's digital literacy skill test can also be used to measure student's skill with very low to excellent categories.

Web-Based Simulation on Physics Learning to Enhance Digital Literacy Skill of High School Students

“…There were 10 open-ended tests prepared by researchers in accordance with the measured aspects of digital literacy. This item was adapted from the development of questions to measure students' digital literacy skill by [36]. Table 2 shows the aspects of each item.…”

Development of an Instrument to Measure Conceptualizations and Competencies About Conversational Agents on the Example of Smart Speakers

“…Other approaches used open-ended questions to assess information ("judging its relevance and purpose"), safety ("personal and data protection"), and problem-solving ("solve conceptual problems through digital means") (e.g., Perdana et al, 2019). Another instrument for measuring digital literacy comes from Ng (2012), who distinguished a technical dimension (technical and operational skills for learning with information and communication technology and using it in everyday life), a cognitive dimension (ability to think critically about searching, evaluating, and creating digital information) and a socialemotional dimension (ability to use the internet responsibly for communication, socializing and learning).…”

“…With its focus on the (future) potential and the reach of smart speakers, CALS four might be less close to attitudes towards technology. Future studies should widen the nomological net and incorporate more diverse constructs and variables into the analysis, such as experiences with technology, technological competencies, or psychological variables associated with technology-related competencies (among others: underlying motivations of usage and non-usage, personality traits such as openness to new experiences, or curiosity) (e.g., Jenkins, 2006;Literat, 2014;Porat et al, 2018;Perdana et al, 2019). Moreover, future studies should also consider associations with behavioral indicators.…”

“…In sum, as the starting point of this study was a specific conceptional framework (Long and Magerko, 2020) we neglected other conceptualizations of digital competencies, which involve additional definitions and domains. Thus, future studies should consider different concepts of digital literacy, AIL, and CAL (e.g., Jenkins, 2006;Literat, 2014;Vuorikari et al, 2016;Porat et al, 2018;Perdana et al, 2019). The approach presented by Ng (2012) might be promising since the social-emotional dimension extended previous measures by considering the interactivity of digital technology.…”

Pathways to a Knowledge Society: A Proposal for a Hierarchical Model for Measuring Digital Literacy Among Israeli pre-service Teachers

“…Yang (2019) asserts that an assessment tool is needed to measure student digital literacy. Perdana, et al (2019a) developed an open-ended test for measuring digital literacy skills of students, which was based on five components of digital literacy skill: information, communication, content creation, safety, and problem-solving -thus, missing some of the essential digital literacy components. In addition, we have identified a number of instruments measuring various aspects of digital literacies: a scale designed to measure self-efficacy for information literacy (Kurbanoglu et al, 2006); a self-report instrument to measure pre-service teachers' ICT competencies in education, comprising competencies to support pupils for ICT use in class, and competencies to use ICT for instructional design (Tondeur et al, 2017); an information literacy test for higher education, which aims to assess different levels of thinking skills (Boh Podgornik et al, 2016); an information literacy test to predict the strength of some attributes of digital nativeness (ICT ownership, ICT experiences, internet confidence and number of ICT-rich university courses) on the information literacy of university students (Šorgo et al, 2017), and a digital literacy scale for teenagers (Rodríguez-de-Dios et al, 2016).…”