Methodology for Evaluating a Simulator's Capability to Support Assessment of Mariner Proficiency

Abstract

The goal of this project was to develop a structured process for evaluating the capability of simulators to support mariner performance assessment. In this approach, simulator evaluation objectives and criteria are based on performance-based mariner assessment requirements. The approach produces a simulator evaluation protocol that addresses four areas of simulator capability: (1) providing flexible exercise programming to the assessor; (2) replicating the characteristics of actual equipment; (3) simulating the operational conditions required to demonstrate the desired mariner performance; and (4) supporting thorough debriefing of the assessment candidate. A case study showed that our evaluation process was feasible. The process proved to be both sensitive and valuable in differentiating the capabilities of two simulators used to support mariner assessment objectives. This approach can be generalized to the evaluation of simulators of equipment used in other industries.

Methodology for Evaluating a Simulator's Capability to Support Assessment of Mariner Proficiency

“…programming, equipment set-up, simulation, and debriefing). Each of the evaluation objectives had between hvo and 15 separate evaluation criteria associated with it (Raby, et al 1999). The criteria represented specific requirements in control and display elements of the simulator.…”