A Framework for Mixing Methods in Quantitative Measurement Development, Validation, and Revision

Abstract

A framework for quantitative measurement development, validation, and revision that incorporates both qualitative and quantitative methods is introduced. It extends and adapts Adcock and Collier's work, and thus, facilitates understanding of quantitative measurement development, validation, and revision as an integrated and cyclical set of procedures best achieved through mixed methods research. It also offers a systematic guide concerning how these procedures may be undertaken through detailing key "stages," "levels," and practical "tasks." A case study illustrates how qualitative and quantitative methods may be mixed through the use of the proposed framework in the cross-cultural content-and construct-related validation and subsequent revision of a quantitative measure.The contribution of this article to mixed methods research literature is briefly discussed.

Beyond traditional understanding of gender measurement: the gender (re)presentation approach

“…As noted above, researchers should be prepared to evaluate evidence of an instrument's content as well as construct validity among the population in which it is to be disseminated. A mixed method approach, drawing upon the strengths of both qualitative and quantitative techniques, should ideally be adopted in order to do so (Luyt 2012b). In cross-cultural research, for example, the equivalence of measures should not be assumed.…”

A Dialectic Data Integration Approach for Mixed Methods Survey Validation

“…[3][4][5] Two flexible frameworks for mixing qualitative and quantitative methods during development and validation have been proposed. 6,7 Despite this growing trend in mixed methods research, several authors have suggested more work is needed in determining appropriate and useful ways methods can be mixed to improve survey development, especially with regard to understanding the concept of validity in mixed methods research. 7,8 These broader methodological questions are of direct interest to the growing field of engineering education research where the mixing of quantitative and qualitative is a growing trend but mixed modes of validation have not been extensively explored.…”

“…Therefore, validation is a dynamic, iterative process that must carefully consider context in addition to theory. 1,6,8,12,14 Messick (1995) acknowledged that all available evidence can and should be utilized when considering the validity of an assessment in a particular context, but also that certain evidence may be stronger than other types. 14 In particular evidence must be introduced to account for six particular aspects of the unified validity construct: content validity, substantive validity, structural validity, generalizability, external aspects of validity, and consequential validity.…”

“…This approach attempts to incorporate qualitative and quantitative data on equal footing and truly integrate the findings rather than consider them in isolation. 6,7 Luyt (2012) applied a convergent parallel mixed methods design 13 to evaluate the use of three versions of a survey, each in a different language, with three different cultural groups. 6 Thematic analysis was used to identify conceptual, contextual, and semantic issues with the survey implementation with samples from the three distinct cultural groups.…”

“…6,7 Luyt (2012) applied a convergent parallel mixed methods design 13 to evaluate the use of three versions of a survey, each in a different language, with three different cultural groups. 6 Thematic analysis was used to identify conceptual, contextual, and semantic issues with the survey implementation with samples from the three distinct cultural groups. These findings were evaluated holistically with quantitative factor analysis and item analysis to evaluate and improve specific survey items.…”