The Effects of Recognition Accuracy and Vocabulary Size of a Speech Recognition System on Task Performance and User Acceptance

Abstract

Industrial Engineering and Operations Research (ABSTRACT)Automatic speech recog n ition systems have at last advanced to the state that they are now a feasible alternative for human-machine communication in selected appli ...cations. As such, research efforts are now beginning to focus on characteristics of the human, the recognition device, and the interface which optimize the system performance, rather than the previous trend of determining factors affecting recognizer performance alone. This study investigated two characteristics of the recognition device, the accuracy level at which it recognizes speech, and the vocabulary size of the recognizer as a percent of task vocabulary size to determine their effects on system per:ormance. In addition, the study considered one characteristic of the user, age. Briefly, subjects performed a data entry task under each of the treatment conditions. Task completion time and the number of errors remaining at the end of each session were recorded. After each session, subjects rated the recognition device used as to its acceptability for the task.The accuracy level at which the recognizer was performing significantly influenced the task completion time as well as the user's acceptability ratings, but had only a small effect on the number of errors left uncorrected. The available vocabulary size also significantly affected the task completion time; however its effect on the final error rate and on the acceptability ratings was negligible. The age of the subject was also found to influence both objective and subjective measures. Older subjects in general required longer times to complete the tasks; however, they consistently rated the speech input systems more favorably than the younger subjects. AcknowledgementsThe author wishes to express appreciation to Dr. Robert D. Dryden for the support and guidance he provided as committee chairman throughout the course of this research. Special thanks are also due to committee members Beverly H. Williges for the direction and expertise she provided, and Professor Paul T. Kemmerling for his advice and encouragement. I also wish to thank Calvin L. Selig for the development of the speech recognition system simulation software.On a more personal note, I would like to thank my parents for a lifetime of support and encouragement. And finally, I would like to thank my husband John, whose patience and encouragement have helped make the past two years an enjoyable as well as rewarding experience. Acknowledgements Iv List of Tables IntroductionAutomatic speech recognition refers to the ability of a machine to discriminate spoken utterances. The first such devices were devel~ped in the mid 1950'st and significant progress has been made within the past decade. It has long been known that unconstrained speech is the fastest, most efficient means for two people to communicate (Chapanis, 1975). Now, applying spoken communication to human-machine systems is proving to be an efficient alternative for many applications. Such applications includ...

Serving Customers With Automatic Speech Recognition-Human-Factors Issues

“…15 Researchers usingan ASR-based data-retrieval task found that olderusers were slower to recover from systemrecognition errors. 16 Suchan effect illustrates the importance ofcarefully considering a user group's demography.…”