Reliability and Validity

Lo, Soo Kien; Lee, S.

doi:10.4324/9781315755649-29

Cited by 30 publications

(14 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 6 shows the scores achieved in the tests. In the development of a quantitative instrument for assessment purposes, it is crucial to measure its quality [95,96], which mainly consists of measuring its validity (if the assessment instrument assesses what should be measured and refers to how test scores are interpreted and used [94]) and reliability (if the assessment instrument produces similar results under equivalent conditions) [97]. To obtain evidence of quality, different methods are available, and studies involving the development of an assessment instrument should include enough evidence [95].…”

Section: Second Iteration With Citizensmentioning

confidence: 99%

Design and Validation of a Novel Tool to Assess Citizens’ Netiquette and Information and Data Literacy Using Interactive Simulations

Bartolomé

Garaizar

2022

Sustainability

View full text Add to dashboard Cite

Until recently, most of the digital literacy frameworks have been based on assessment frameworks used by commercial entities. The release of the DigComp framework has allowed the development of tailored implementations for the evaluation of digital competence. However, the majority of these digital literacy frameworks are based on self-assessments, measuring only low-order cognitive skills. This paper reports on a study to develop and validate an assessment instrument, including interactive simulations to assess citizens’ digital competence. These formats are particularly important for the evaluation of complex cognitive constructs such as digital competence. Additionally, we selected two different approaches for designing the tests based on their scope, at the competence or competence area level. Their overall and dimensional validity and reliability were analysed. We summarise the issues addressed in each phase and key points to consider in new implementations. For both approaches, items present satisfactory difficulty and discrimination indicators. Validity was ensured through expert validation, and the Rasch analysis revealed good EAP/PV reliabilities. Therefore, the tests have sound psychometric properties that make them reliable and valid instruments for measuring digital competence. This paper contributes to an increasing number of tools designed to evaluate digital competence and highlights the necessity of measuring higher-order cognitive skills.

show abstract

Section: Second Iteration With Citizensmentioning

confidence: 99%

Design and Validation of a Novel Tool to Assess Citizens’ Netiquette and Information and Data Literacy Using Interactive Simulations

Bartolomé

Garaizar

2022

Sustainability

View full text Add to dashboard Cite

show abstract

“…When developing a quantitative instrument, it is important to assess its quality [7,28,29]. This is a process that mainly depends on assessing its reliability and validity [29].…”

Section: Quality Of Quantitative Instrumentsmentioning

confidence: 99%

“…From a psychometric perspective, the first property states whether the instrument provides the same (or similar) results under similar conditions or inputs, while the second one states whether it measures what is supposed to be measured [28]. Several methods to conduct these assessments have been proposed in the past [7] and have been classified into different categories. For instance, the detailed review provided by [30] identified four types of both reliability and validity assessments as shown in Table 2.…”

Section: Quality Of Quantitative Instrumentsmentioning

confidence: 99%

“…As a consequence, it largely depends on the quality of the employed instrument. Regardless of the structure of the instrument, its quality is given by the fulfillment degree of two psychometric properties: reliability and validity [7]. As recent reviews showed [8,9], the topic of digital competence in higher education comprises of a large and fertile body of studies.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

On the quality of quantitative instruments to measure digital competence in higher education: A systematic mapping study

2021

View full text Add to dashboard Cite

In this study, we report on a Systematic Mapping Study (SMS) on how the quality of the quantitative instruments used to measure digital competencies in higher education is assured. 73 primary studies were selected from the published literature in the last 10 years in order to 1) characterize the literature, 2) evaluate the reporting practice of quality assessments, and 3) analyze which variables explain such reporting practices. The results indicate that most of the studies focused on medium to large samples of European university students, who attended social science programs. Ad hoc, self-reported questionnaires measuring various digital competence areas were the most commonly used method for data collection. The studies were mostly published in low tier journals. 36% of the studies did not report any quality assessment, while less than 50% covered both groups of reliability and validity assessments at the same time. In general, the studies had a moderate to high depth of evidence on the assessments performed. We found that studies in which several areas of digital competence were measured were more likely to report quality assessments. In addition, we estimate that the probability of finding studies with acceptable or good reporting practices increases over time.

show abstract

“…Prior to disseminating the survey, items were analyzed for validity, clarity, and relevance by content experts. Reliability coefficients were calculated, evaluated, and found to be .8 or greater for all Likert-scaled items (item 14,  = .907; item 15,  = .898; item 16,  = .878; item 17,  = .827), indicating acceptable reliability (Mueller & Knapp, 2019).…”

Section: Self-perceptions Of Preparedness For Remote Continuity Of Learning Instrumentmentioning

confidence: 99%

Remote continuity of learning and the COVID-19 pandemic: educators self-perceptions of preparedness

Ceballos

Vitale

Gordon

2021

JPSP

View full text Add to dashboard Cite

In February 2020, the COVID-19 pandemic upended education systems as schools suspended in-person instruction and transitioned to remote learning to mitigate the spread of the disease within their communities. As school systems implemented remote learning, we sought to examine differences in teacher and educational leader self-perceptions of preparedness for remote continuity of learning and communicating with stakeholders, factors contributing to preparedness, and recommended needs in future educational leadership preparation. We used a survey design to complete this research by distributing the Self-perceptions of Preparedness for Remote Continuity of Learning Instrument (SPRCLI) © to a convenience sample of teachers and educational leaders enrolled in a graduate-level educational leadership program. Eighty teachers and 15 educational leaders completed the SPRCLI © survey. Analysis indicated differences in self-perceptions of preparedness to communicate with stakeholders with educational leaders demonstrating higher perceptions of communication preparedness. Contributing factors for preparedness for remote continuity of learning included colleagues, professional development, prior technology experience, and experiences with online learning. Participants recommended future educational leadership preparation include professional development and coursework on digital applications, best practices for remote learning, and development of school plans for remote continuity of learning. This study contributes to an understanding of teacher and educational leader preparedness for remote learning at the start of the COVID-19 crisis.

show abstract

Reliability and Validity

Cited by 30 publications

References 34 publications

Design and Validation of a Novel Tool to Assess Citizens’ Netiquette and Information and Data Literacy Using Interactive Simulations

Design and Validation of a Novel Tool to Assess Citizens’ Netiquette and Information and Data Literacy Using Interactive Simulations

On the quality of quantitative instruments to measure digital competence in higher education: A systematic mapping study

Remote continuity of learning and the COVID-19 pandemic: educators self-perceptions of preparedness

Contact Info

Product

Resources

About