Speech Analysis for Fatigue and Sleepiness Detection of a Pilot

Vasconcelos, Carla Aparecida de; Vieira, Maurílio Nunes; Kecklund, Göran; Yehia, Hani Camille

doi:10.3357/amhp.5134.2019

Cited by 19 publications

(10 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several different technologies have been studied in order to assess their useability, of which electrocardiography and eye metrics are two major areas of interest ( Hu and Lodewijks, 2020 ). Promising alternatives are speech analysis, which detects subtle changes in voice patterns, and photoplethysmogram sensors in the aviation headset, which detect changes in heart rate ( de Vasconcelos et al, 2019 ; Wilson et al, 2020 ).…”

Section: Operational Countermeasuresmentioning

confidence: 99%

Fatigue in Aviation: Safety Risks, Preventive Strategies and Pharmacological Interventions

Wingelaar-Jagt

Wingelaar²,

Riedel

et al. 2021

Front. Physiol.

View full text Add to dashboard Cite

Fatigue poses an important safety risk to civil and military aviation. In addition to decreasing performance in-flight (chronic) fatigue has negative long-term health effects. Possible causes of fatigue include sleep loss, extended time awake, circadian phase irregularities and work load. Despite regulations limiting flight time and enabling optimal rostering, fatigue cannot be prevented completely. Especially in military operations, where limits may be extended due to operational necessities, it is impossible to rely solely on regulations to prevent fatigue. Fatigue management, consisting of preventive strategies and operational countermeasures, such as pre-flight naps and pharmaceuticals that either promote adequate sleep (hypnotics or chronobiotics) or enhance performance (stimulants), may be required to mitigate fatigue in challenging (military) aviation operations. This review describes the pathophysiology, epidemiology and effects of fatigue and its impact on aviation, as well as several aspects of fatigue management and recommendations for future research in this field.

show abstract

Section: Operational Countermeasuresmentioning

confidence: 99%

Fatigue in Aviation: Safety Risks, Preventive Strategies and Pharmacological Interventions

Wingelaar-Jagt

Wingelaar²,

Riedel

et al. 2021

Front. Physiol.

View full text Add to dashboard Cite

show abstract

“…Objective speech assessment is more accurate, replicable, and feasible than perceptual analysis ( Noffs et al, 2018 ). Related studies have reported that changes in the characteristic parameters of the voice pattern are closely related to fatigue levels ( de Vasconcelos et al, 2019 ). Our results also suggest that the corresponding parameters change with alterations in the fatigue level, and the characteristic parameters of the vocal pattern can characterize the fatigue state.…”

Section: Discussionmentioning

confidence: 99%

A rapid, non-invasive method for fatigue detection based on voice information

Gao¹,

Ma²,

Yang³

et al. 2022

Front. Cell Dev. Biol.

View full text Add to dashboard Cite

Fatigue results from a series of physiological and psychological changes due to continuous energy consumption. It can affect the physiological states of operators, thereby reducing their labor capacity. Fatigue can also reduce efficiency and, in serious cases, cause severe accidents. In addition, it can trigger pathological-related changes. By establishing appropriate methods to closely monitor the fatigue status of personnel and relieve the fatigue on time, operation-related injuries can be reduced. Existing fatigue detection methods mostly include subjective methods, such as fatigue scales, or those involving the use of professional instruments, which are more demanding for operators and cannot detect fatigue levels in real time. Speech contains information that can be used as acoustic biomarkers to monitor physiological and psychological statuses. In this study, we constructed a fatigue model based on the method of sleep deprivation by collecting various physiological indexes, such as P300 and glucocorticoid level in saliva, as well as fatigue questionnaires filled by 15 participants under different fatigue procedures and graded the fatigue levels accordingly. We then extracted the speech features at different instances and constructed a model to match the speech features and the degree of fatigue using a machine learning algorithm. Thus, we established a method to rapidly judge the degree of fatigue based on speech. The accuracy of the judgment based on unitary voice could reach 94%, whereas that based on long speech could reach 81%. Our fatigue detection method based on acoustic information can easily and rapidly determine the fatigue levels of the participants. This method can operate in real time and is non-invasive and efficient. Moreover, it can be combined with the advantages of information technology and big data to expand its applicability.

show abstract

“…Whitmore and Fisher [28] conducted a 36-hour experiment to obtain speech and fatigue data, and via statistical analysis, they proved that fundamental frequency is a valid indicator for fatigue. De Vasconcelos, et al [15] took speech analysis as an accident investigation tool. After comparing voice patterns of non-sleepy data and voice data one hour before the accident, they found pauses during speaking and elocution articulation rates are connected with fatigue.…”

Section: B Voice Patternsmentioning

confidence: 99%

“…Like electrooculogram (EOG), electrocardiogram (ECG), electroencephalogram (EEG), and functional near-infrared spectroscopy (fNIRS), sensors are reliable in measuring human cognitive states [14]. Nevertheless, preceding sensors can disturb the operator's regular duty, for their intrusive resulted in restrictions on the head or hand movements, or uncomfortably sensor wearing experience [15]. Unobtrusively data-collecting technologies are more suitable for exploring the tower controller's behavior patterns than invasive devices.…”

Section: Introductionmentioning

confidence: 99%

Integrating Ergonomics Into Safety Management: A Conceptual Risk Assessment Model for Tower Controllers at Multiple Altitudes

et al. 2021

View full text Add to dashboard Cite

Plateau and high plateau airports aggravate tower controllers' vulnerability, facing high safety risks, yet the risk evaluation paradigm to manage safety at multiple altitudes still lacks. The study aimed to investigate the effects of altitudes on controllers' mental workload and fatigue to assess the safety risk and introduced voices, mental workload, and fatigue, into a conceptual risk assessment model. Controllers from the Civil Aviation Administration of China (CAAC) conducted the experimental tasks, reporting mental workload and perception fatigue across three altitudes: 0 m, 2243 m, and 3569.7 m. With experimental data: this research (1) quantitatively compared the voice feature differences with feature engineering, and an image quality measure, (2) explored the effects of altitude, sleep, and fatigue, (3) tested the effects of altitude and task complexity on mental workload, and (4) evaluated the airport safety risks under ergonomic factors. Notably, the study revealed that Log-Mel spectrograms outperformed Mel Frequency Cepstral Coefficients (MFCC) in severe fatigue detection. Altitude and task complexity had significant main effects on the mental workload, but altitude had no significant moderator effects on the relationship between sleep and fatigue. The simulation results show that under the low task complexity, the operation risk is low over three airport elevations (with the human error rate < 10 −3 ), whereas under the high task complexity, the operation risk increased with altitudes (from 1.73× 10 −3 to 1.02 × 10 −1 ). Together, these results suggest that ergonomic factors influenced airport safety risk at multiple altitudes and promising real-time fatigue detection with voice features at different altitudes.

show abstract

Speech Analysis for Fatigue and Sleepiness Detection of a Pilot

Cited by 19 publications

References 0 publications

Fatigue in Aviation: Safety Risks, Preventive Strategies and Pharmacological Interventions

Fatigue in Aviation: Safety Risks, Preventive Strategies and Pharmacological Interventions

A rapid, non-invasive method for fatigue detection based on voice information

Integrating Ergonomics Into Safety Management: A Conceptual Risk Assessment Model for Tower Controllers at Multiple Altitudes

Contact Info

Product

Resources

About