Content Validity Using a Mixed Methods Approach

Abstract

There is paucity of detailed information in the literature on how to develop procedures for estimating content validity to increase trustworthiness of assessment instruments. The authors present three unique examples, interpretation, and application of Tables of Specifications (ToS) for generating these estimates. The development and logic of the ToS requires presenting evidence that has transparency and creates trustworthiness in the validity estimates by maintaining an audit trail and using of triangulation, expert debriefing, and peer review. The argument presented is that content validity requires a mixed methods approach since data are developed through qualitative and quantitative methods that inform each other. This process is iterative and provides feedback on the effectiveness of the ToS through a process of determining consensus.

Design and Implementation Content Validity Study: Development of an instrument for measuring Patient-Centered Communication

“…Therefore, validity evidence should be obtained on each study for which an instrument is used. 4 Content validity, also known as definition validity and logical validity, 5 can be defined as the ability of the selected items to reflect…”

Establishing a Baseline Measurement of Intercultural Competence of Ukrainian Faculty: Setting the Stage for Domestic and Global Intercultural Understanding

“…While the purpose of this study was to establish a baseline of psychometric estimates of intercultural competence for the population of Ukrainian faculty members, future studies on validity and reliability of the GPI are needed to further adjust the survey to the researched population and to the construct of intercultural competence. The crossvalidation of the GPI using the Table of Specification (Newman, Lim, & Pineda, 2013) and factor analysis may be used to refine the research instrument to measure more components of intercultural competence (Deardorff, 2006;. It is also important to conduct more studies on the population of Ukrainian faculty members to add more items to the modified scales in the second modified version of the instrument.…”

“…Using a Table of Specifications (Newman, Lim, & Pineda, 2013), two out of four content area experts (raters) thought that there were not enough scale items to measure the construct of intercultural competence, with Scales 3 and 6 of the first modified version of the GPI being of the most concern to them. The researcher addressed the issue of validity and reliability of the GPI by taking into consideration the raters' feedback, performing two sets of factor analyses on 100% and 50% of the sample and revising the instrument accordingly.…”

“…To find out if the six scales of the GPI measure the concept of intercultural competence sufficiently, the researcher used a Table of Specifications' (ToS) template developed by Newman, Lim, and Pineda (2013). The columns in the ToS represented the six scales of the GPI and rows listed the components of intercultural competence outlined in Deardorff (2006).…”

“…Because the research instrument has neither been previously used as a measure of intercultural competence, nor administered in Ukraine, it was cross-validated using a Table of Specification (Newman, Lim, & Pineda, 2013) and two sets of factor analyses. As a result, a modified version of the GPI was created for use in Ukraine.…”