Making Music More Accessible for Cochlear Implant Listeners: Recent Developments

Nogueira, Waldo; Nagathil, Anil; Martin, Rainer

doi:10.1109/msp.2018.2874059

Cited by 32 publications

(20 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For this reason, it is important to expose the VIR parameter of the source separation algorithm such that the CI listener can adjust it to its own needs. Here one can foresee that the wireless communication to smartphones or the use of remote controls with such a parameter exposed could be very beneficial to make music more accessible for CI users (Nogueira et al, 2019).…”

Section: Discussionmentioning

confidence: 99%

Design and Evaluation of a Real-Time Audio Source Separation Algorithm to Remix Music for Cochlear Implant Users

2020

Self Cite

View full text Add to dashboard Cite

A cochlear implant (CI) is a surgically implanted electronic device that partially restores hearing to people suffering from profound hearing loss. Although CI users, in general, obtain a very good reception of continuous speech in the absence of background noise, they face severe limitations in the context of music perception and appreciation. The main reasons for these limitations are related to channel interactions created by the broad spread of electrical fields in the cochlea and to the low number of electrodes that stimulate it. Moreover, CIs have severe limitations when it comes to transmitting the temporal fine structure of acoustic signals, and hence, these devices elicit poor pitch and timber perception. For these reasons, several signal processing algorithms have been proposed to make music more accessible for CI users, trying to reduce the complexity of music signals or remixing them to enhance certain components, such as the lead singing voice. In this work, a deep neural network that performs real-time audio source separation to remix music for CI users is presented. The implementation is based on multi-layer perception (MLP) and has been evaluated using objective instrumental measurements to ensure clean source estimation. Furthermore, experiments in 10 normal hearing (NH) and 13 CI users to investigate how the vocals to instruments ratio (VIR) set by the tested listeners were affected in realistic environments with and without visual information. The objective instrumental results fulfill the benchmark reported in previous studies by introducing distortions that are shown to not be perceived by CI users. Moreover, the implemented model was optimized to perform real-time source separation. The experimental results show that CI users prefer vocals 8 dB enhanced with the respect to the instruments independent of acoustic sound scenarios and visual information. In contrast, NH listeners did not prefer a VIR different than zero dB.

show abstract

Section: Discussionmentioning

confidence: 99%

Design and Evaluation of a Real-Time Audio Source Separation Algorithm to Remix Music for Cochlear Implant Users

2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…The Auditory Distortion Ratio (ADR) measure proposed by Nagathil et al (2016) was developed to instrumentally evaluate changes due to reduced frequency selectivity and spectral spread between processed and unprocessed signals (see also Nogueira et al, 2019). It is based on broadened auditory filters that mimick the reduced frequency selectivity in listeners with severe hearing loss.…”

Section: Methodsmentioning

confidence: 99%

“…In contrast to pitch-related musical features, the perception of rhythmic features by CI users is comparable to normal-hearing (NH) listeners (McDermott, 2004, Looi et al, 2012, Limb and Roy, 2014). Rhythmic information can be defined as regular temporal patterns with periodicities ranging from 50 ms to 5 s (Nogueira et al, 2019) and thus varies significantly slower than the temporal fluctuations being perceived as pitch. Rhythm is encoded in the slowly varying temporal envelope of pulse trains for the individual electrodes.…”

Section: Introductionmentioning

confidence: 99%

Interactive Evaluation of a Music Preprocessing Scheme for Cochlear Implants Based on Spectral Complexity Reduction

et al. 2019

Self Cite

View full text Add to dashboard Cite

Music is difficult to access for the majority of CI users as the reduced dynamic range and poor spectral resolution in cochlear implants (CI), amongst others constraints, severely impair their auditory perception. The reduction of spectral complexity is therefore a promising means to facilitate music enjoyment for CI listeners. We evaluate a spectral complexity reduction method for music signals based on principal component analysis that enforces spectral sparsity, emphasizes the melody contour and attenuates interfering accompanying voices. To cover a wide range of spectral complexity reduction levels a new experimental design for listening experiments was introduced. It allows CI users to select the preferred level of spectral complexity reduction interactively and in real-time. Ten adult CI recipients with post-lingual bilateral profound sensorineural hearing loss and CI experience of at least 6 months were enrolled in the study. In eight consecutive sessions over a period of 4 weeks they were asked to choose their preferred version out of 10 different complexity settings for a total number of 16 recordings of classical western chamber music. As the experiments were performed in consecutive sessions we also studied a potential long term effect. Therefore, we investigated the hypothesis that repeated engagement with music signals of reduced spectral complexity leads to a habituation effect which allows CI users to deal with music signals of increasing complexity. Questionnaires and tests about music listening habits and musical abilities complemented these experiments. The participants significantly preferred signals with high spectral complexity reduction levels over the unprocessed versions. While the results of earlier studies comprising only two preselected complexity levels were generally confirmed, this study revealed a tendency toward a selection of even higher spectral complexity reduction levels. Therefore, spectral complexity reduction for music signals is a useful strategy to enhance music enjoyment for CI users. Although there is evidence for a habituation effect in some subjects, such an effect has not been significant in general.

show abstract

“…Nogueira, Nagathil y Martin en su estudio enfocado al desarrollo de la música más accesible para las personas con sordera, mencionan que debido a que la música es un medio universal de interacción sociocultural y además desencadenador de emociones positivas, las personas con acceso limitado a la música como la comunidad sorda, enfrentan una degradación considerable en su calidad de vida, su formación personal, su creatividad, su temple, entre otras (Nogueira, Nagathil, y Martin, 2019). Así, pese a que algunos miembros de esta comunidad no se encuentran interesados en tener acceso a la música, muchos sí quieren ser incluidos como una cuestión de equidad y participación (Baijal, Kim, Branje, Russo, y Fels, 2012).…”

Section: Problemática Sobre Las Personas Con Discapacidad Auditiva Y La Músicaunclassified

Desarrollo de la apreciación musical en niños Sordos mediante estimulación por vibraciones desde una aplicación móvil

Cortazar

Rojano-Cáceres

2021

RBIE

View full text Add to dashboard Cite

La música como un fenómeno acústico, se encuentra relacionada estrechamente con el sentido del oído, motivo por el cual posiblemente resulte difícil concebir que las personas con discapacidad (PCD) auditiva profunda o personas sordas puedan ser capaces de comprender los conceptos asociados a dicho fenómeno. Para una persona que vive sin discapacidad auditiva el acceder a los componentes musicales y desarrollarse en muchos casos en dicha área resulta natural. Sin embargo la enseñanza de la música y sus conceptos por métodos tradicionales no resulta factible para el caso de PCD auditiva. Para ese caso en particular será mejor recurrir a otros sentidos, como puede ser el caso del sentido del tacto. Por lo tanto, en el presente artículo de investigación se describe el proceso a través del cual se aborda la enseñanza de conceptos asociados a la música en un grupo de niños sordos a través de una aplicación móvil, empleando la estimulación a través de las vibraciones generadas por los teléfonos móviles. Sin embargo, debido a las condiciones imperantes de la pandemia causada por COVID-19, la investigación debió llevarse de forma remota teniendo la posibilidad de monitorear solo a una muestra de los niños del grupo original, mediante el seguimiento de las interacciones con la aplicación.

show abstract

Making Music More Accessible for Cochlear Implant Listeners: Recent Developments

Cited by 32 publications

References 36 publications

Design and Evaluation of a Real-Time Audio Source Separation Algorithm to Remix Music for Cochlear Implant Users

Design and Evaluation of a Real-Time Audio Source Separation Algorithm to Remix Music for Cochlear Implant Users

Interactive Evaluation of a Music Preprocessing Scheme for Cochlear Implants Based on Spectral Complexity Reduction

Desarrollo de la apreciación musical en niños Sordos mediante estimulación por vibraciones desde una aplicación móvil

Contact Info

Product

Resources

About