Validation Practices in Counseling: Major Journals, Mattering Instruments, and the Kuder Occupational Interest Survey (KOIS)

Chan, Eric K. H.; Munro, David; Huang, Alexander H. S.; Zumbo, Bruno D.; Vojdanijahromi, Roya; Ark, Neelam

doi:10.1007/978-3-319-07794-9_5

Cited by 6 publications

(2 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Section A collected descriptive information about articles and sample characteristics, and noted which articles identified as a review by their synthesis of literature, and Section B collected information about validity and validation evidence. Using a similar coding process as Chan et al (2014) and Cizek, Rosenberg, and Koons (2008), Section B was organized using the following scheme and the five sources identified in the Standards (AERA et al, 2014): (a) test content, (b) relations with other variables, (c) internal structure, (d) consequences, and (e) response processes (see the Appendix for a sample of the coding sheet and definitions). As well, an "other validity" category was included to account for validity sources not described in the Standards (AERA et al, 2014), and reliability was also included, since reliability is a necessary condition for validity (Hubley & Zumbo, 2013).…”

Section: Codingmentioning

confidence: 99%

A Systematic Review of Validation Practices for the Goal Attainment Scaling Measure

Shankar

Marshall

Zumbo

2019

Journal of Psychoeducational Assessment

Self Cite

View full text Add to dashboard Cite

Goal attainment scaling (GAS) is an internationally recognized measure that is widely used in educational, counseling, and clinical settings to identify and evaluate relevant goals for an individual. The GAS is an unusual measure because its content, which consists of goals, is formed by the respondent and/or users in the process of completing the GAS. Using the unified view of validity as a guiding framework, this systematic review examines validation practices and how goals are represented in this measure. This review demonstrates that validation practices tend to focus on aspects that do not support the overall construct validity of the measure, as well as reference to the GAS measure or GAS scores as a property. Several gaps in validity evidence and the various ways goals are conceptualized are described and discussed. The varying ways goals are considered suggest clarity is needed to enhance explanations and score meaning. This review urges researchers to consider ways validity and validation evidence can help verify the many claims that are made about this measure. Future validity research needs to consider application of a theoretical framework and response processes as key aspects of substantiating the construct measured by the GAS.

show abstract

Section: Codingmentioning

confidence: 99%

A Systematic Review of Validation Practices for the Goal Attainment Scaling Measure

Shankar

Marshall

Zumbo

2019

Journal of Psychoeducational Assessment

Self Cite

View full text Add to dashboard Cite

show abstract

“…Quantitative procedures then come into play as the battery of items is tested against statistical criteria, such as Cronbach's alpha coefficient to demonstrate internal consistency [8] or bivariate correlation coefficients to demonstrate construct-related and criterion-related validity [9,10]. Despite the fact that both qualitative and quantitative procedures are involved in instrument development [11,12], quantitative methods may be overemphasised [13] and qualitative methods neglected [14].…”

Section: Introductionmentioning

confidence: 99%

Mixed methods instrument validation: Evaluation procedures for practitioners developed from the validation of the Swiss Instrument for Evaluating Interprofessional Collaboration

Grand-Guillaume-Perrenoud

Geese

Uhlmann

et al. 2023

BMC Health Serv Res

View full text Add to dashboard Cite

Background Quantitative and qualitative procedures are necessary components of instrument development and assessment. However, validation studies conventionally emphasise quantitative assessments while neglecting qualitative procedures. Applying both methods in a mixed methods design provides additional insights into instrument quality and more rigorous validity evidence. Drawing from an extensive review of the methodological and applied validation literature on mixed methods, we showcase our use of mixed methods for validation which applied the quality criteria of congruence, convergence, and credibility on data collected with an instrument measuring interprofessional collaboration in the context of Swiss healthcare, named the Swiss Instrument for Evaluating Interprofessional Collaboration. Methods We employ a convergent parallel mixed methods design to analyse quantitative and qualitative questionnaire data. Data were collected from staff, supervisors, and patients of a university hospital and regional hospitals in the German and Italian speaking regions of Switzerland. We compare quantitative ratings and qualitative comments to evaluate the quality criteria of congruence, convergence, and credibility, which together form part of an instrument’s construct validity evidence. Results Questionnaires from 435 staff, 133 supervisors, and 189 patients were collected. Analysis of congruence potentially provides explanations why respondents’ comments are off topic. Convergence between quantitative ratings and qualitative comments can be interpreted as an indication of convergent validity. Credibility provides a summary evaluation of instrument quality. These quality criteria provide evidence that questions were understood as intended, provide construct validity, and also point to potential item quality issues. Conclusions Mixed methods provide alternative means of collecting construct validity evidence. Our suggested procedures can be easily applied on empirical data and allow the congruence, convergence, and credibility of questionnaire items to be evaluated. The described procedures provide an efficient means of enhancing the rigor of an instrument and can be used alone or in conjunction with traditional quantitative psychometric approaches.

show abstract

Response Processes in the Context of Validity: Setting the Stage

Hubley

Zumbo

2017

Understanding and Investigating Response Processes in Validation Research

View full text Add to dashboard Cite

Validation Practices in Counseling: Major Journals, Mattering Instruments, and the Kuder Occupational Interest Survey (KOIS)

Cited by 6 publications

References 70 publications

A Systematic Review of Validation Practices for the Goal Attainment Scaling Measure

A Systematic Review of Validation Practices for the Goal Attainment Scaling Measure

Mixed methods instrument validation: Evaluation procedures for practitioners developed from the validation of the Swiss Instrument for Evaluating Interprofessional Collaboration

Response Processes in the Context of Validity: Setting the Stage

Contact Info

Product

Resources

About