Human-in-the-Loop Investigation of Variable Separation Standards in the En Route Air Traffic Control Environment

Sollenberger, Randy L.; Hale, Mark

doi:10.1177/1071181311551014

Cited by 2 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A common tool for workload assessment used at the FAA William J. Hughes Technical Center (WJHTC) is the Air Traffic Workload Input Technique (ATWIT; Stein, 1985). The ATWIT scale provides a real-time assessment of workload in air traffic control (ATC) simulations (Ahlstrom & Friedman-Berg, 2006;Endsley, Mogford, Allendoerfer, Snyder, & Stein, 1997;Manning et al, 2002;McAnulty et al, 2005;Pfleiderer, 2005;Sollenberger & Hale, 2011;Sollenberger et al, 1997;Sollenberger & Stein, 1995;Truitt, McAnulty, & Willems, 2004;Truitt, & Muldoon, 2010;Willems et al, 1999;Willems & Heiney, 2002). However, some applications of the ATWIT have deviated from the original scale.…”

Section: Introductionmentioning

confidence: 99%

“…In addition, studies have shown that controller workload is associated with weather-related stressors (Ahlstrom & Friedman-Berg, 2006), situational awareness (Endsley & Rodgers, 1997), and poor runway approach (Stein, 1989). Workload is also an effective tool to support the development of systems and standards (Allendoerfer, Galushka, Mogford, 2000;McAnulty, Zingale, & Willems, 2005;Sollenberger & Hale, 2011).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Scale Size of the Air Traffic Workload Input Technique (ATWIT)

Fincannon

Ahlstrom

2016

Proceedings of the Human Factors and Ergonomics Society Annual

View full text Add to dashboard Cite

Objective: This paper uses a review of previous studies to provide a recommendation for the optimal scale size of the Air Traffic Workload Input Technique (ATWIT). Background: The ATWIT is a measure of workload that was originally a 10-point scale, but subsequent research includes a 7-point variation of this scale. Scale size is known to impact assessment reliability, and more reliable scales produce stronger effect sizes and reduce costs that are associated with experimentation. Therefore, it is important to know whether the 7-point or 10-point version of the scale is more reliable. Method: The authors conducted a preliminary meta-analysis of 15 studies. The analysis examined correlations between ratings using the ATWIT and aircraft count (an objective measure of difficulty) to compare effect sizes across studies with a 7-point scale and a 10-point scale. Results: Findings indicated that the strength of the correlation between ATWIT ratings and aircraft count was greater for the 10-point version of the ATWIT than for the 7-point version. Conclusion: The 10-point scale appears to be more appropriate for the ATWIT. However, the authors recommend further research to examine and control for the effects of potential confounds.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Scale Size of the Air Traffic Workload Input Technique (ATWIT)

Fincannon

Ahlstrom

2016

Proceedings of the Human Factors and Ergonomics Society Annual

View full text Add to dashboard Cite

show abstract

“…Two human-in-the-loop (HITL) simulations have been conducted so far in this program. The first (Sollenberger & Hale, 2011;Sollenberger, Willems, DiRico, Hale, & Deshmukh, 2010) investigated the effect of variable separation requirements within a sector and among aircraft, as well as a variety of automation tools to aid controllers' implementation of variable requirements. The second (Zingale, Willems, Schulz, & Higgins 2012) evaluated issues including (a) the location and format of the Conflict Probe notification on the controller workstation, (b) the replacement of current controller workstation displays with a 30-inch off-the-shelf monitor, and (c) alternative pointing devices to the existing controller workstation trackball (Higgins, Willems, Johnson, & Zingale 2012).…”

mentioning

confidence: 99%

Accuracy Assessment of Air Traffic Conflict Probe Prototype for Operational Evaluations

Masalonis

Rein

Messina³

et al. 2013

Proceedings of the Human Factors and Ergonomics Society Annual

View full text Add to dashboard Cite

This study is part of the Separation Management research program, whose goals include improving the FAA's operational Conflict Probe function. Conflict Probe alerts air traffic controllers to conflicts, or situations where aircraft will be too close to each other. The present study is one link in a chain of research efforts. We used the results of a meta-analysis of Human Factors literature on automation accuracy (Rein, Masalonis, Messina, & Willems, in press) in conjunction with FAA mathematical studies on the accuracy of the current Conflict Probe prototype (Crowell, Fabian, Young, Musialek, & Paglione, 2011; Crowell & Young, 2012) and determined the acceptability of the prototype's conflict detection performance. The present results will feed upcoming operational research including a human-in-the loop (HITL) simulation in which the prototype will be used, by helping establish whether the prototype was "good enough" to improve joint human-automation system performance. In addition, the present analysis enhances the methodology for determining the accuracy of the operational Conflict Probe, although for this paper we did not evaluate or report on operational data. We obtained data from the aforementioned FAA mathematical analysis, which reported the prototype's performance on some of the standard SDT metrics. We further analyzed their data to generate values on a wider set of accuracy metrics, and compared the results to the findings of Rein et al. regarding how accurate automation "should" be, as well as considering the results from an operational/face validity perspective. We focused on reliability, a measure of overall percent correct by the automation, which has been used in past multi-experiment analyses of automation accuracy (Wickens & Dixon, 2007), and which Rein et al. found to have a relationship to system performance. With a "best case" estimate of Conflict Probe reliability, its performance far exceeds that needed to improve system performance. However, the estimate may have been too liberal from an operational perspective, because the input data included many correct rejections where the proximity of the aircraft was well beyond the distances defining a conflict. In such cases, the automation's failure to alert would have been technically correct, but not useful to the controller, who would know without any automated assistance that no conflict was present. We conclude that the current Conflict Probe prototype is suitable for conducting the HITL research, but that additional scenario evaluation research should be run to determine for what kinds of conflicts and near-conflicts the automation can complement rather than duplicate controller skill. This scenario evaluation research, and related follow-up mathematical analysis, will answer the question "how accurate is Conflict Probe?" The HITL will answer "how accurate does Conflict Probe need to be?" These answers will be evaluated in conjunction with each other to improve the operational Conflict Probe.

show abstract

Human-in-the-Loop Investigation of Variable Separation Standards in the En Route Air Traffic Control Environment

Cited by 2 publications

References 0 publications

Scale Size of the Air Traffic Workload Input Technique (ATWIT)

Scale Size of the Air Traffic Workload Input Technique (ATWIT)

Accuracy Assessment of Air Traffic Conflict Probe Prototype for Operational Evaluations

Contact Info

Product

Resources

About