Sonification – an Alternative Presentation of the Electrocardiogram: A Systematic Literature Review

Kanev, Ivan; Iliev, Ivo; Krasteva, Vessela

doi:10.1109/et.2019.8878650

Cited by 3 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The application of realtime monitoring, such as medical monitoring, critical infrastructure monitoring, alarms, or real‐time feedback on body movement, is vast and distinct from the purpose of data exploration. Especially with respect to sonification and auditory display, the field is well‐researched (e.g., [KIK19,SJMT19,VRGM20,WMY*17,HRM15]), and we will not cover such designs in this STAR. Artistic contributions have the potential to be highly inspirational for our field but require a different search method and, most likely, a different system of classification.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Open Your Ears and Take a Look: A State‐of‐the‐Art Report on the Integration of Sonification and Visualization

Enge,

Elmquist,

Caiola

et al. 2024

Computer Graphics Forum

View full text Add to dashboard Cite

The research communities studying visualization and sonification for data display and analysis share exceptionally similar goals, essentially making data of any kind interpretable to humans. One community does so by using visual representations of data, and the other community employs auditory (non‐speech) representations of data. While the two communities have a lot in common, they developed mostly in parallel over the course of the last few decades. With this STAR, we discuss a collection of work that bridges the borders of the two communities, hence a collection of work that aims to integrate the two techniques into one form of audiovisual display, which we argue to be “more than the sum of the two.” We introduce and motivate a classification system applicable to such audiovisual displays and categorize a corpus of 57 academic publications that appeared between 2011 and 2023 in categories such as reading level, dataset type, or evaluation system, to mention a few. The corpus also enables a meta‐analysis of the field, including regularly occurring design patterns such as type of visualization and sonification techniques, or the use of visual and auditory channels, showing an overall diverse field with different designs. An analysis of a co‐author network of the field shows individual teams without many interconnections. The body of work covered in this STAR also relates to three adjacent topics: audiovisual monitoring, accessibility, and audiovisual data art. These three topics are discussed individually in addition to the systematically conducted part of this research. The findings of this report may be used by researchers from both fields to understand the potentials and challenges of such integrated designs while hopefully inspiring them to collaborate with experts from the respective other field.

show abstract

Section: Methodsmentioning

confidence: 99%

“…(Non‐)systematic literature reviews have been published in several of the mentioned areas [WMY*17,KIK19,SMS21,HRM15]. To the best knowledge of the authors, no comprehensive STAR covering the entire spectrum of audiovisual or related multimodal monitoring applications and approaches has been conducted thus far.…”

Section: Adjacent Topicsmentioning

confidence: 99%

Open Your Ears and Take a Look: A State‐of‐the‐Art Report on the Integration of Sonification and Visualization

Enge,

Elmquist,

Caiola

et al. 2024

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…By mapping these features to basic auditory parameters, such as pitch, volume, and rhythm, healthcare providers or patients can listen to the ECG data and identify abnormalities or changes in the heart’s electrical activity. For the purpose of intuitive human interpretation of sonified ECG signals, two main application fields were identified [ 20 , 21 ]: Audio interpretation of abnormal heart rate values or rhythmic patterns; Mapping of ECG parameters for better audio representation and human perception. …”

Section: Introductionmentioning

confidence: 99%

Application of Convolutional Neural Network for Decoding of 12-Lead Electrocardiogram from a Frequency-Modulated Audio Stream (Sonified ECG)

Krasteva,

Iliev,

Tabakov

2024

Sensors

View full text Add to dashboard Cite

Research of novel biosignal modalities with application to remote patient monitoring is a subject of state-of-the-art developments. This study is focused on sonified ECG modality, which can be transmitted as an acoustic wave and received by GSM (Global System for Mobile Communications) microphones. Thus, the wireless connection between the patient module and the cloud server can be provided over an audio channel, such as a standard telephone call or audio message. Patients, especially the elderly or visually impaired, can benefit from ECG sonification because the wireless interface is readily available, facilitating the communication and transmission of secure ECG data from the patient monitoring device to the remote server. The aim of this study is to develop an AI-driven algorithm for 12-lead ECG sonification to support diagnostic reliability in the signal processing chain of the audio ECG stream. Our methods present the design of two algorithms: (1) a transformer (ECG-to-Audio) based on the frequency modulation (FM) of eight independent ECG leads in the very low frequency band (300–2700 Hz); and (2) a transformer (Audio-to-ECG) based on a four-layer 1D convolutional neural network (CNN) to decode the audio ECG stream (10 s @ 11 kHz) to the original eight-lead ECG (10 s @ 250 Hz). The CNN model is trained in unsupervised regression mode, searching for the minimum error between the transformed and original ECG signals. The results are reported using the PTB-XL 12-lead ECG database (21,837 recordings), split 50:50 for training and test. The quality of FM-modulated ECG audio is monitored by short-time Fourier transform, and examples are illustrated in this paper and supplementary audio files. The errors of the reconstructed ECG are estimated by a popular ECG diagnostic toolbox. They are substantially low in all ECG leads: amplitude error (quartile range RMSE = 3–7 μV, PRD = 2–5.2%), QRS detector (Se, PPV > 99.7%), P-QRS-T fiducial points’ time deviation (<2 ms). Low errors generalized across diverse patients and arrhythmias are a testament to the efficacy of the developments. They support 12-lead ECG sonification as a wireless interface to provide reliable data for diagnostic measurements by automated tools or medical experts.

show abstract

“…An electrocardiogram is a visual representation in the form of a graph depicting voltage versus time of the electrical activity of the heart, which can be measured using electrodes placed on the skin. Two main research directions can be identified in sonification research on ECGs (see for a systematic review on the topic): 1) sonifications of temporal fea [25] tures and 2) sonifications of certain pathological states corresponding to changes in waves that compose the ECG signal, i.e. processes serving as supporting tools in diagnosis tasks ].…”

Section: Introductionmentioning

confidence: 99%

Mapping Inter-Cardiovascular Time-Frequency Coherence to Harmonic Tension in Sonification of Ensemble Interaction Between a Covid-19 Patient and the Medical Team

Frid¹,

Orini²,

Martinelli³

et al. 2021

Proceedings of the 26th International Conference on Auditory Display (ICAD 2021)

View full text Add to dashboard Cite

This paper presents exploratory work on sonic and visual representations of heartbeats of a COVID-19 patient and a medical team. The aim of this work is to sonify heart signals to reflect how a medical team comes together during a COVID-19 treatment, i.e. to highlight other aspects of the COVID-19 pandemic than those usually portrayed through sonification, which often focuses on the number of cases. The proposed framework highlights synergies between sound and heart signals through mapping between time-frequency coherence (TFC) of heart signals and harmonic tension and dissonance in music. Results from a listening experiment suggested that the proposed mapping between TFC and harmonic tension was successful in terms of communicating low versus high coherence between heart signals, with an overall accuracy of 69%, which was significantly higher than chance. In the light of the performed work, we discuss how links between heart- and sound signals can be further explored through sonification to promote understanding of aspects related to cardiovascular health.

show abstract

Sonification – an Alternative Presentation of the Electrocardiogram: A Systematic Literature Review

Cited by 3 publications

References 24 publications

Open Your Ears and Take a Look: A State‐of‐the‐Art Report on the Integration of Sonification and Visualization

Open Your Ears and Take a Look: A State‐of‐the‐Art Report on the Integration of Sonification and Visualization

Application of Convolutional Neural Network for Decoding of 12-Lead Electrocardiogram from a Frequency-Modulated Audio Stream (Sonified ECG)

Mapping Inter-Cardiovascular Time-Frequency Coherence to Harmonic Tension in Sonification of Ensemble Interaction Between a Covid-19 Patient and the Medical Team

Contact Info

Product

Resources

About