Validating the use of Anchoring Vignettes for the Correction of Response Scale Differences in Subjective Questions

Abstract

Comparing self-assessed indicators of subjective outcomes such as health, work disability, political efficacy, job satisfaction, etc. across countries or socio-economic groups is often hampered by the fact that different groups use different response scales. This paper develops an integrated framework in which objective measurements are used to validate vignette-based corrections. The framework is applied to objective and subjective self-assessments of drinking behavior by students in Ireland. Model comparisons using the Akaike information criterion favor a specification with response consistency and vignette corrected response scales. Put differently, vignette based corrections appear quite effective in bringing objective and subjective measures closer together.

Systematic measurement error in self-reported health: is anchoring vignettes the way out?

“…Kapteyn et al (2011) using an Internet-based panel in the USA find a mixed evidence on the response consistency assumption which holds for certain health domains and not in others. Van Soest et al (2011) develop an integrated framework in which objective measurements are used to validate vignette-based corrections of subjective assessments of drinking behavior by students in Ireland. Bago d 'Uva et al (2011) point that the assumption of response consistency is testable given sufficiently comprehensive objective indicators that independently identify response scales.…”

Subjektivní hodnocení problémů s pohybem: Užití parametrického modelu metody ukotvujících vinět

“…Před využitím této metody a formulací ukotvujících vinět v nových oblastech lze tudíž všem výzkum-níkům doporučit nejen nastudování základů CHOPIT modelu, ale také prostudování literatury ke splnění předpokladů metody (např. Bago d 'Uva et al, 2011;Kapteyn et al, 2011;van Soest et al, 2011;Voňková & Hullegie, 2011;van Soest & Voňková, 2013).…”

Non-cognitive skills and individual earnings: new evidence from PIAAC

“…Without a common metric, individual responses are not likely to be interpersonally comparable (see King et al 2004;Soest et al 2011). In the case of PIAAC, there is the additional problem that the above questions might provide incomplete information on individuals' noncognitive skills.…”

“…In contrast, non-cognitive skills and personality traits are less present in internationally comparable datasets. First, because the measurement of non-cognitive skills has been traditionally based on self-reports which raises several methodological concerns due to misreports, difficulties in interpersonal comparisons or the socially desirable response bias (see King et al 2004;Paulhus 1984;Soest et al 2011). Second, because attention to these types of skills is relatively new and started to grow at the beginning of the twentyfirst century with the works by Heckman and his coauthors.…”