Situation awareness based on eye movements in relation to the task environment

Abstract

The topic of situation awareness has received continuing interest over the last decades. Freeze-probe methods, such as the Situation Awareness Global Assessment Technique (SAGAT), are commonly employed for measuring situation awareness. The aim of this paper was to review validity issues of the SAGAT and examine whether eye movements are a promising alternative for measuring situation awareness. First, we outlined six problems of freeze-probe methods, such as the fact that freeze-probe methods rely on what the operator has been able to remember and then explicitly recall. We propose an operationalization of situation awareness based on the eye movements of the person in relation to their task environment to circumvent shortfalls of memory mediation and task interruption. Next, we analyzed experimental data in which participants (N = 86) were tasked to observe a display of six dials for about 10 min, and press the space bar if a dial pointer crossed a threshold value. Every 90 s, the screen was blanked and participants had to report the state of the dials on a paper sheet. We assessed correlations of participants' task performance (% of threshold crossing detected) with visual sampling scores (% of dials glanced at during threshold crossings) and freeze-probe scores. Results showed that the visual-sampling score correlated with task performance at the threshold-crossing level (r = 0.31) and at the individual level (r = 0.78). Freeze-probe scores were low and showed weak associations with task performance. We conclude that the outlined limitations of the SAGAT impede measurement of situation awareness, which can be computed more effectively from eye movement measurements in relation to the state of the task environment. The present findings have practical value, as advances in eye-tracking cameras and ubiquitous computing lessen the need for interruptive tests such as SAGAT. Eye-based situation awareness is a predictor of performance, with the advantage that it is applicable through real-time feedback technologies.

Physiological Measurements of Situation Awareness: A Systematic Review

“…In some different studies, although no correlation between eye tracking metrics and direct SA measures was found, researchers offer that eye tracking metrics can be more beneficial in predicting performance in SA tasks compared with direct methods such as the SAGAT. For example, de Winter et al (2019) reported that the visual sampling score was found to be positively related to performance ratings, whereas In this study, the authors reported that "entropy" increased with CARS scores (p = .06). the correlation between the SAGAT score and performance rating was much less pronounced.…”

A Systematic Review and Meta-Analysis of Direct Objective Measures of Situation Awareness: A Comparison of SAGAT and SPAM

“…This analysis shows that claims that SAGAT is intrusive (Chiappe et al, 2012; de Winter et al, 2019; Durso et al, 1998; Salmon et al, 2011; Sarter & Woods, 1991) are unwarranted, with no negative effects on performance found in the 11 studies that examined it. The conditions of SAGAT administration are significantly different than interruptions with unrelated tasks.…”

“…Some researchers have examined the processes that people use to develop SA using measures such as eye tracking (de Winter, Eisma, Cabrall, Hancock, & Stanton, 2019; Ikuma, Harvey, Taylor, & Handal, 2014; Smolensky, 1993), communications (Bolstad et al, 2007; Gorman, Cooke, Pederson, Connor, & DeJoode, 2005; Orasanu, 2000; Prince, Salas, & Stout, 1995), verbal protocols (Hall & Phelps, 1983; Rose, Bearman, Naweed, & Dorrian, 2019; Sullivan & Blackman, 1991; Walker, Stanton, & Young, 2008), and physiological measurement (Wilson, 2000). While process measures can provide insights into how people develop SA, they can only be used to indirectly infer the quality and completeness of the resulting SA, as a state of knowledge about the situation, obtained by the individuals involved.…”

How do pedestrians distribute their visual attention when walking through a parking garage? An eye-tracking study

“…The fact that pedestrians are inclined to look at the wheels of cars appears to be a novel finding in the literature. Exceptions are Aupetit, Espié, and Bouaziz (2015), who noted that motorcyclists look at the front wheels of cars to infer their intended lateral movement, and Vlakveld (2014; see also De Winter, Eisma, Cabrall, Hancock, & Stanton, 2019), who used turned front wheels as a hazard precursor in a hazard perception study.…”