Errors in Scoring Objective Personality Tests

Abstract

Scoring objective personality tests is considered clerical, and presumably, straightforward in nature. This may be the reason that few studies, if any, have investigated the impact of scoring error on widely used tests, such as the MMPI or 801. Errors, even if infrequent (e.g. as few as 1 % of tests), may adversely affect many hundreds or thousands of tests administered annually, however. In a study of three popular tests taken from three independent settings, this study found that the interpretation of popular tests are vulnerable even to small errors (i.e., 1 or 2 misscored items per test). This study explored the influence of two factors, scoring procedure complexity and commitment to scoring accuracy, hypothesized to be related to the occurence of scoring error with fewer errors occuring when higher commitment to accuracy and lower scoring procedure complexity are present. The scoring procedure complexity effect was predicted to be subordinate to the commitment to accuracy effect. Three popular tests were sampled from three different settings and rescored to check for accuracy.Twenty-one percent of tests scored with low commitment to accruacy were erroneous, while tests scored with full commitment to accuracy had 1 % errors.Scoring procedure complexity, categorized as high and low, yielded 29% and 14% erroneous tests, respectively, in the less than full commitment to accuracy sample, and 0 and 4% in the full commitment to accuracy sample. The results provide strong support for the factors as major predictors of scoring error, as well as the interaction effect anticipated. Other risk factors, such as commercial computer scoring errors and lack of agreement on test scoring stand~rds , were II also found to distort scores. The frequency and severity of erroneous findings in this study, the author argues, are unlikely to be specific to this study, but instead more general. The author shows how awareness of the two factors, as well as other sources of error, can be used to reduce the risk of scoring error and offers practical recommendations to improve scoring accuracy.iii Acknowledgments

Advancing Personality Assessment Terminology: Time to Retire "Objective" and "Projective" As Personality Test Descriptors

“…One problem is that these positive connotations are not fully warranted for the inventories to which they typically refer. Scoring errors are certainly one potential concern (e.g., Allard & Faust, 2000). More substantively, however, if the kind of self-report scales that are classified as objective actually were "objective" in a meaningful sense of that word, then there would not be such a huge literature examining the various response styles and biases that affect scores derived from these instruments.…”

A before and after study on personality assessment in adolescents exposed to the 2009 earthquake in L'Aquila, Italy: influence of sports practice

“…The questionnaires were administered individually by the same professional psychologists who administered the questionnaires before the earthquake, all of whom received specific training on the MMPI-A. 16 Adequate matching of the subjects was ensured by the experimenters. Exclusion criteria consisted of protocols with a VRIN T-score greater than 74 (considered inconsistent) and protocols containing more than 30 unanswered items.…”

Psychological and Legal Aspects of Dangerous Sex Offenders: A Review of the Literature

“…While no measure is completely accurate -possibly owing to errors in test development or administration (Allard & Faust, 2000) -they are generally accurate enough to be effective in estimating risk of violence. Given this, it is therefore surprising that legislation does not demand that they are exclusively used in determining 'unacceptable risk'.…”