Faster Command Input Using the Multimodal Controller Working Position “TriControl”

Ohneiser, Oliver; Jauer, Malte-Levin; Rein, Jonathan R.; Wallace, Matt

doi:10.3390/aerospace5020054

Cited by 3 publications

(8 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These modalities can be used to input the three basic ATC command parts, i.e., aircraft callsign, command type, and value, into the ATC system [9]. Hence, conventional subsequent command parts that are uttered verbally are replaced by parallel ATC system input with different modalities [10].…”

Section: Of 26mentioning

confidence: 99%

“…First analyses with the multimodal CWP prototype TriControl showed an acceleration of command input by up to 15% [10]. Furthermore, the artificial voice broadcast or data-link transmission of commands resulting from combined command parts of the parallel ATC system input in TriControl can also reduce misunderstandings in verbal communication caused by various "foreign language" English accents [11], that might even lead to serious accidents [12].…”

Section: Of 26mentioning

confidence: 99%

“…Hence, the command could be sent to the aircraft via datalink or could be read by an artificial voice via the usual radio telephony channel. More details on the background of TriControl as well as functionalities especially with respect to aspects of command element input orders and timing can be found in [10]. .…”

Section: Multimodal Cwp Tricontrolmentioning

confidence: 99%

“…The term effectiveness does not say anything about the efficiency of TriControl. The potential for efficiency gains-comparing TriControl with pure speech commands and following manual ATC system input-has been reported in [10].…”

Section: Ratings On Input Procedures (I)mentioning

confidence: 99%

See 3 more Smart Citations

Operational Feasibility Analysis of the Multimodal Controller Working Position “TriControl”

et al. 2020

Self Cite

View full text Add to dashboard Cite

Current Air Traffic Controller working positions (CWPs) are reaching their capacity owing to increasing levels of air traffic. The multimodal CWP prototype TriControl combines automatic speech recognition, multitouch gestures, and eye-tracking, aiming for more natural and improved human interaction with air traffic control systems. However, the prototype has not yet undergone systematic evaluation with respect to feasibility. This paper evaluates the operational feasibility, focusing on the system usability of the approach CWP TriControl and its fulfillment of operational requirements. Fourteen controllers took part in a simulation study to evaluate the TriControl concept. The active approach controllers among the group of participants served as the main core target subgroup. The ratings of all controllers in the TriControl assessment were, on average, generally in slight agreement, with just a few showing statistical significance. However, the active approach controllers performed better and rated the system much more positively. The active approach controllers were strongly positive regarding the system usability and acceptance of this early-stage prototype. Particularly, ease of use, user-friendliness, and learnability were perceived very positively. Overall, they were also satisfied with the command input procedure, and would use it for their daily work. Thus, the participating controllers encourage further enhancements to be made to TriControl.

show abstract

Section: Of 26mentioning

confidence: 99%

Section: Of 26mentioning

confidence: 99%

Section: Multimodal Cwp Tricontrolmentioning

confidence: 99%

Section: Ratings On Input Procedures (I)mentioning

confidence: 99%

See 2 more Smart Citations

Operational Feasibility Analysis of the Multimodal Controller Working Position “TriControl”

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Benefits of multimodal and more natural interaction at a controller working position (CWP) have already been investigated, i.e., to combine interaction technologies such as speech recognition and eye-tracking with each other to support ATCo tasks [7]. Hence, integrating further unobtrusive sensor data from eye-and mouse-tracking with ABSR and reasonably using these modalities' benefits promises to further improve efficiency of ATCos' CWP interaction.…”

Section: Introductionmentioning

confidence: 99%

Integrating Eye- and Mouse-Tracking with Assistant Based Speech Recognition for Interaction at Controller Working Positions

2021

Self Cite

View full text Add to dashboard Cite

Assistant based speech recognition (ABSR) prototypes for air traffic controllers have demonstrated to reduce controller workload and aircraft flight times as a result. However, two aspects of ABSR could enhance benefits, i.e., (1) the predicted controller commands that speech recognition engines use can be more accurate, and (2) the confirmation process of ABSR recognition output, such as callsigns, command types, and values by the controller, can be less intrusive. Both tasks can be supported by unobtrusive eye- and mouse-tracking when using operators’ gaze and interaction data. First, probabilities for predicted commands should consider controllers’ visual focus on the situation data display. Controllers will more likely give commands to aircraft that they focus on or where there was a mouse interaction on the display. Furthermore, they will more likely give certain command types depending on the characteristics of multiple aircraft being scanned. Second, it can be determined via eye-tracking instead of additional mouse clicks if the displayed ABSR output has been checked by the controller and remains uncorrected for a certain amount of time. Then, the output is assumed to be correct and is usable by other air traffic control systems, e.g., short-term conflict alert. If the ABSR output remains unchecked, an attention guidance functionality triggers different escalation levels to display visual cues. In a one-shot experimental case study with two controllers for the two implemented techniques, (1) command prediction probabilities improved by a factor of four, (2) prediction error rates based on an accuracy metric for three most-probable aircraft decreased by a factor of 25 when combining eye- and mouse-tracking data, and (3) visual confirmation of ABSR output promises to be an alternative for manual confirmation.

show abstract

A Socio-Technical Simulation Model for the Design of the Future Single Pilot Cockpit: An Opportunity to Improve Pilot Performance

Piera

Muñoz²,

Gil

et al. 2022

IEEE Access

View full text Add to dashboard Cite

The future deployment of single pilot operations must be supported by new cockpit computer services. Such services require an adaptive context-aware integration of technical functionalities with the concurrent tasks that a pilot must deal with. Advanced artificial intelligence supporting services and improved communication capabilities are the key enabling technologies that will render future cockpits more integrated with the present digitalized air traffic management system. However, an issue in the integration of such technologies is the lack of socio-technical analysis in the design of these teaming mechanisms. A key factor in determining how and when a service support should be provided is the dynamic evolution of pilot workload. This paper investigates how the socio-technical model-based systems engineering approach paves the way for the design of a digital assistant framework by formalizing this workload. The model was validated in an Airbus A-320 cockpit simulator, and the results confirmed the degraded pilot behavioral model and the performance impact according to different contextual flight deck information. This study contributes to practical knowledge for designing human-machine task-sharing systems.

show abstract

Faster Command Input Using the Multimodal Controller Working Position “TriControl”

Cited by 3 publications

References 20 publications

Operational Feasibility Analysis of the Multimodal Controller Working Position “TriControl”

Operational Feasibility Analysis of the Multimodal Controller Working Position “TriControl”

Integrating Eye- and Mouse-Tracking with Assistant Based Speech Recognition for Interaction at Controller Working Positions

A Socio-Technical Simulation Model for the Design of the Future Single Pilot Cockpit: An Opportunity to Improve Pilot Performance

Contact Info

Product

Resources

About