Visual Attention Assessment for Expert-in-the-Loop Training in a Maritime Operation Simulator

Abstract

Improving the training programs for maritime operations is beneficial to enhance maritime safety in practice. In this paper, we propose a novel approach to the assessment of visual attention in a maritime operation so as to support an expertin-the-loop training program. Experts' knowledge of maritime operation and experiences in the simulator are incorporated into the training program in three ways. First, through a questionnaire, information about task division, identification of critical operation, and definitions of areas of interest (AOIs) is incorporated as prior knowledge for modeling visual attention. Second, a weight scale factor that emphasizes the high importance of visual focus in critical operations is utilized to generate an operations-dependent attention map. Third, based on an expert's attention map and visual switch between AOIs, a similarity metric is designed as a comprehensive evaluation between saliency and visual transition. A case study of heavy lifting operation is performed by two groups of trainees who have received different briefings about "critical operation". The assessment result shows the group with more detailed briefing obtains a 6% similarity score higher than the other group, which is consistent with the debriefing result of a superior performance of that group. The proposed approach is thus verified effective in assessing visual attention for the expert-in-the-loop training program.

Assessment of the Potential of Wrist-Worn Wearable Sensors for Driver Drowsiness Detection

“…Due to advancements in camera technology in combination with novel approaches in computer vision and image processing, camera-based drowsiness detection has been receiving more and more attention in recent years [18]. These methods evaluate mainly three parameters: eye movements (eye blinking and eye closure activity) via eye-tracking, that was also investigated for usage in maritime operations and aviation [19][20][21], facial expressions (yawning, jaw drop, brow rise, and lip stretch), and head position (head scaling/nodding) [22]. In particular, many studies focused on the use of machine (deep) learning-based approaches [23][24][25][26][27].…”

A Survey of Eye Tracking in Automobile and Aviation Studies: Implications for Eye-Tracking Studies in Marine Operations

“…It also suggests that the attention map method and clustering algorithm method are good alternates to allow AOI definition through data analysis. More specifically, for stimuli that have unpredictable high fixation areas such as marine crane lifting [25], mooring system replacement, ship maneuvering, and wind turbine installation, the expert-defined method is the most appropriate. When the stimuli are easy to separate, or the high fixation areas are easy to estimate such as towing, dynamic positioning [118], navigation [26], [27], and operation of underwater robots such as a remotely operated vehicle, the stimulus-generated defined method is the more suitable approach.…”

“…Human error also affects marine operations. The growth of demanding marine operations, such as offshore petroleum extraction, subsea pipeline deployment, and wind turbine installation, has increased the threat human error poses to the safety of people and property [25]. Attempts have been made to use eye tracking to assess marine operations [25]- [28], but systematic instructions for using the method and conducting experiments and assessing its effectiveness remain sparse.…”

Cloud Model-Based Intelligent Evaluation Method in Marine Engine Room Simulator

“…Aiming at the problems of subjectivity and uncertainty when making combat capability assessment plans, Wang et al [19] combined expert experience and a self-learning algorithm to propose an adaptive fuzzy wavelet neural network collaborative combat capability evaluation model. Li et al [20] combined the expert knowledge and experience in training plans through a questionnaire, weight enhancement and similarity calculation and proposed a new method to evaluate the visual attention in maritime action. Based on the decision-making chain of UAV cooperative combat, Huang et al [21] established an evaluation model of UAV collaborative combat effectiveness by considering multiple collaboration capabilities.…”