This paper presents the design and evaluation of four sonification methods to support monitoring and diagnosis in Electrocardiography (ECG). In particular we focus on an ECG abnormality called ST-elevation which is an important indicator of a myocardial infarction. Since myocardial infarction represents a life-threatening condition it is of essential value to detect an ST-elevation as early as possible. As part of the evaluated sound designs, we propose two novel sonifications: (i) Polarity sonification, a continuous parameter-mapping sonification using a formant synthesizer and (ii) Stethoscope sonification, a combination of the ECG signal and a stethoscope recording. The other two designs, (iii) the water ambience sonification and the (iv) morph sonification, were presented in our previous work about ECG sonification (Aldana Blanco AL, Steffen G, Thomas H (2016) In: Proceedings of Interactive Sonification Workshop (ISon). Bielefeld, Germany). The study evaluates three components across the proposed sonifications (1) detection performance, meaning if participants are able to detect a transition from healthy to unhealthy states, (2) classification accuracy, that evaluates if participants can accurately classify the severity of the pathology, and (3) aesthetics and usability (pleasantness, informativeness and long-term listening). The study results show that the polarity design had the highest accuracy rates in the detection task whereas the stethoscope sonification obtained the better score in the classification assignment. Concerning aesthetics, the water ambience sonification was regarded as the most pleasant. Furthermore, we found a significant difference between sound/music experts and non-experts in terms of the error rates obtained in the detection task using the morph sonification and also in the classification task using the stethoscope sonification. Overall, the group of experts obtained lower error rates than the group of non-experts, which means that further training could improve accuracy rates and, particularly for designs that rely mainly on pitch variations, additional training is needed in the non-experts group.

Audio-visual speech enhancement is the task of improving the quality of a speech signal when video of the speaker is available. It opens-up the opportunity of improving speech intelligibility in adverse listening scenarios that are currently too challenging for audio-only speech enhancement models. The Audio-Visual Speech Enhancement (AVSE) challenge aims to set the first benchmark in this area. We provide participants with datasets and scripts to test their audio-visual speech enhancement models under a common framework for both training and evaluation. The data is derived from real-world videos, and comprises noisy mixes, in which audio from target speaker is mixed with either a competing speaker or a noise signal. The submitted systems are evaluated by conducting AV intelligibility tests involving human participants. We expect this challenge to be a platform for advancing the field of audio-visual speech-enhancement and to provide further insight about the scope and limitations of current AV speech enhancement approaches.

No abstract

No abstract

This paper is a continuation and extension of our previous work [1]. CardioSounds is a portable system that allows users to measure and sonify their electrocardiogram signal in real-time. The ECG signal is acquired using the hardware platform BITalino and subsequently analyzed and sonified using a Raspberry Pi. Users can control basic features from the system (start recording, stop recording) using their smartphone. The system is meant to be used for diagnostic and monitoring of cardiac pathologies, providing users with the possibility to monitor a signal without occupying their visual attention. In this paper, we introduce a novel method, anticipatory mapping, to sonify rhythm disturbances such as Atrial Fibrillation, Atrial flutter and Ventricular Fibrillation. Anticipatory mapping enhances perception of rhythmic details without disrupting the direct perception of the actual heart beat rhythm. We test the method on selected pathological data involving three of the most known rhythm disturbances. A preliminary perception test to assess aesthetics of the sonifications and its possible use in medical scenarios shows that the anticipatory mapping method is regarded as informative discerning healthy and pathological states, however there is no agreement about a preferred sonification type.