Models, Measures, and Judgments in System Design

Abstract

This paper assumes increasing use of analytical models in system design. Some characteristics of such models and requirements for human performance data compatible with them are discussed. Methods of obtaining human performance data for use in design models are considered. The use of expert judges to generate performance measures is reviewed. Two new studies are reported in support of the proposition that expert judgments may offer a practical method of obtaining performance measure with potentially wide application in analytical modeling efforts.

Validity of Expert Judgments of Performance Time

“…A previous paper (Knowles, Burger, Mitchell, Hanifan, and Wulfeck, 1969) addressed to the increasing use of analytical models in system design discussed some characteristics of such models and their requirements for human performance data. Methods of obtaining human performance data for use in analytical models were considered, and the use of expert judges to generate performance measures was reviewed.…”

“…Those findings suggested that the statistical reliability of performance reliability estimates could be improved by developing techniques to identify and capitalize on the systematic effects among tasks and judges (Knowles, et al, 1969).…”