Integrated Research Paradigm for Complex Experimentation

Abstract

Most human factors design problems involve large data spaces with so many factors that a single experiment investigating all of these factors simultaneously is unreasonable. The design of a computer-based telephone inquiry system is presented as an example interface which required consideration of over 100 independent variables which could affect end-user performance. An integrated research paradigm involving the three major steps of selecting, describing, and optimizing these independent variables is presented. Specific methods and design implications are described for each of these steps. This overall strategy for sequential experimentation is discussed in terms of its usefulness in human factors design applications.

Human Factors Experimental Design and Analysis Reference

“…Williges and Williges (1989) proposed a paradigm for sequential research in human factors that involves the three-step approach shown on this slide.…”

“…For example, there were 14 potential factors to consider in designing the harmonic structure of the speech output system. Step 1 in the Williges and Williges (1989) paradigm summarized on a previous slide dealt with selecting the major independent variables that would be investigated through sequential experimentation. Both nonexperimental and experimental procedures were used to select these independent variables.…”

“…Both coded values and real-world levels of the 10 factors are shown on the slide. Step 2 of the Williges and Williges (1989) paradigm as shown on a previous slide deals with describing inter-relationships among the 10 factors selected in Step 1.…”

“…These additional data points can be combined with the data collected in the preceding set of small experiments to form the database for building an integrated empirical model that can include all the main factors of interest. This slide summarizes the series of sequential studies in the example problem that were conducted in Step 2 of the Williges and Williges (1989) paradigm. The 10 factors in the example problem consisted of four continuous factors and six dichotomous factors.…”

“…The coded values of the 10 factors for these six additional data points are listed at the bottom of this slide. The final aspect of Step 2 of the Williges and Williges (1989) paradigm that was shown on a previous slide is to build integrated empirical models based on all the data collected in the sequential studies. This slide summarizes three integrated models of all 10 factors in the example problem that separately represent the three major dependent variables, Total Search Time (TST), User Added Key Presses (UAK), and Transcription Accuracy (TA).…”