Anchoring Vignettes

Abstract

Abstract. Individuals differ in the way they use rating scales to describe themselves, and these differences are particularly pronounced in children and early adolescents. One promising remedy is to correct (or ''anchor'') an individual's responses according to the way they use the scale when they rate an anchoring vignette (a set of hypothetical targets differing on the attribute of interest). Studying adolescents' self-reports of their socio-emotional attributes, we compared traditional self-report scores with vignette-corrected scores in terms of reliability (internal consistency), discriminant validity (scale intercorrelations), and criterion validity (predicting achievement test scores in language and math). A large and representative sample of 12th grade Brazilian students (N = 8,582, 62% female, mean age 18.2) were administered a Portugueselanguage self-report inventory assessing social-emotional skills related to the Big Five personality dimensions. Correcting scores according to vignette ratings led to increases in the reliability of scales measuring Conscientiousness and Openness, but discriminant validity and criterion validity increased only when each scale was corrected using its own corresponding vignette set. Moreover, accuracy in rating the vignettes was correlated with language achievement test scores, suggesting that verbal factors play a role in providing both normative vignette ratings of others and self-reports that are reliable and valid.

Effects of Anchoring Vignettes on Comparability and Predictive Validity of Student Self-Reports in 64 Cultures

“…However, the individual-level correlations of rescaled scale scores of one particular scale based on target set of vignettes and nontarget set of vignettes ranged from .440 to .482, and the country-level correlations ranged from .509 to .721. The correlations among different sets of vignettes are not strong enough to conclude that one set of vignettes would work for the rescaling of different target constructs, which speaks against any one-size-fits-all application of vignettes (e.g., Primi, Zanon, Santos, De Fruyt, & John, 2016).…”

Some Contributions of Economics to the Study of Personality

“…The unadjusted negative relationship was the misleading result of reference bias. In addition, Primi et al (2016) found that anchoring vignettes improved internal consistency (reliability) for measures of socioemotional skills, that is, it improved the extent to which a group of items hung together. Source: The conscientiousness ranks come from Schmitt et al (2007).…”

Short Version of Self-Assessment Scale of Job Performance

“…We control the response bias and acquiescence using four inverted items (two related to the task and two to the context) besides anchoring vignettes. A study by Primi et al (2016) indicated that the anchoring vignettes help in the control of the self-style response. Thus, for this study, three vignettes were created, describing examples of individuals with high, medium, and low job performance.…”

“…Moreover, as is well known, the responses given to this type of self-reported scale are influenced by the subject's response style, which can lead to bias in the research results that use them (Primi et al, 2016). One threat is the acquiescence, also known as the "yea-saying" effect, which concept is related to the inclination to endorse positive categories of a Likert scale despite the item content.…”