Two Quantitative Approaches for Estimating Content Validity

Abstract

Instrument content validity is often established through qualitative expert reviews, yet quantitative analysis of reviewer agreements is also advocated in the literature. Two quantitative approaches to content validity estimations were compared and contrasted using a newly developed instrument called the Osteoporosis Risk Assessment Tool (ORAT). Data obtained from a panel of eight expert judges were analyzed. A Content Validity Index (CVI) initially determined that only one item lacked interrater proportion agreement about its relevance to the instrument as a whole (CVI = 0.57). Concern that higher proportion agreement ratings might be due to random chance stimulated further analysis using a multirater kappa coefficient of agreement. An additional seven items had low kappas, ranging from 0.29 to 0.48 and indicating poor agreement among the experts. The findings supported the elimination or revision of eight items. Pros and cons to using both proportion agreement and kappa coefficient analysis are examined.

Content Validation of the Community College Student Success Program Inventory

“…The Community College Student Success Program Inventory (CC-SSPI), a protocol for structured interviews of program personnel, was developed to fill this gap and serve as a tool for researchers and practitioners alike to account for critical features of student success programs in detailed and comparable ways. The CCSSPI was the result of an extensive literature review and multiple empirical studies (Center for Community College Student Engagement [CCCSE], 2012;Hatch, 2016;Hatch & Bohlig, 2015, 2016Hatch, Mardock-Uman, Garcia, & Johnson, in press The purpose of this study was a content validation (Wynd, Schmidt, & Schaefer, 2003) of the CCSSPI to determine the extent to which the instrument adequately accounts for its domain of interest: in this case, the goals, curricular and pedagogical features, and logistical mechanisms that characterize student success programs in comparable ways. In recognition of the socio-cultural nature of student success programs that ultimately are designed to make explicit often unspoken norms and expectations of college-going as a form of learned social literacy (Gildersleeve, 2010;Hatch, 2016), the CCSSPI relies on the framework of activity theory (Engeström, 2000;Leont'ev, 1978) to specify which features to account for.…”

Incorporación de paciente virtual en portafolio de estudiantes de medicina de pregrado

“…Se realizó una comparación entre la calificación final del PCC y del curso en general, ambas graduadas con escala de 1 a 7. En ambos casos el punto de corte de aprobación se estableció mediante el método de Angoff modificado, estimándose por parte de un panel de docentes involucrados en la corrección de PCC, el puntaje con el cual un estudiante demuestra las mínimas competencias necesarias para su aprobación [26][27][28] .…”

Validação de conteúdo de indicadores de qualidade para avaliação do cuidado de enfermagem

“…The determination of the true validity of measurements is reached by means of the empirical research, grounded in a systematic exam of conceptual abstractions and performed through a process of observation and estimation of responses, aimed at identifying and explaining a phenomenon of interest (10) .…”