Diagnostic scales are essential to the health and social sciences, and to the individuals that provide the data. Although statistical models for scale data have been researched for decades, it remains nearly universal that scale scores are sums of weights assigned a priori to question choice options (sum scores), respectively. We propose several modifications of psychometric testing theory that together demonstrate remarkable improvements in the quality of rating scale scores. Our model represents performance as a space with a metric structure by transforming probability into surprisal or information. The estimation algorithm permits the analysis of data from tens and hundreds of thousands of test takers in a few minutes on consumer level computing equipment. Standard errors of performance estimates are shown to be as small as a quarter of those of sum scores. Open access software resources are presented.

Ramsay and Wiberg used a new version of item response theory that represents test performance over nonnegative closed intervals such as [0, 100] or [0, n] and demonstrated that optimal scoring of binary test data yielded substantial improvements in point-wise root-mean-squared error and bias over number right or sum scoring. We extend these results by showing that optimal scoring of the full information in option choices produces about as much further improvement in these measures of score performance as was achieved by going from sum scoring to optimal binary scoring.