Voice conversion in cochlear implantation

Wilkinson, Eric P.; Abdel-Hamid, Ossama; Galvin, John J.; Jiang, Hui; Fu, Qiuyun

doi:10.1002/lary.23744

Cited by 8 publications

(5 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This may be partly due to poor representation and/or perception of voice characteristics. Previous studies have shown that CI users' gender categorization performance is highly variable and generally poorer than that of NH listeners (Fu et al 2004(Fu et al , 2005Balaban 2009, 2010;Massida et al 2013;Wilkinson et al 2013). It was argued in these studies that CI users might rely more on F0 than NH listeners.…”

Section: Introductionmentioning

confidence: 97%

Gender Categorization Is Abnormal in Cochlear Implant Users

Fuller¹,

Gaudrain²,

Clarke³

et al. 2014

JARO

Self Cite

112

View full text Add to dashboard Cite

In normal hearing (NH), the perception of the gender of a speaker is strongly affected by two anatomically related vocal characteristics: the fundamental frequency (F0), related to vocal pitch, and the vocal tract length (VTL), related to the height of the speaker. Previous studies on gender categorization in cochlear implant (CI) users found that performance was variable, with few CI users performing at the level of NH listeners. Data collected with recorded speech produced by multiple talkers suggests that CI users might rely more on F0 and less on VTL than NH listeners. However, because VTL cannot be accurately estimated from recordings, it is difficult to know how VTL contributes to gender categorization. In the present study, speech was synthesized to systematically vary F0, VTL, or both. Gender categorization was measured in CI users, as well as in NH participants listening to unprocessed (only synthesized) and vocoded (and synthesized) speech. Perceptual weights for F0 and VTL were derived from the performance data. With unprocessed speech, NH listeners used both cues (normalized perceptual weight: F0=3.76, VTL=5.56). With vocoded speech, NH listeners still made use of both cues but less efficiently (normalized perceptual weight: F0=1.68, VTL=0.63). CI users relied almost exclusively on F0 while VTL perception was profoundly impaired (normalized perceptual weight: F0=6.88, VTL=0.59). As a result, CI users' gender categorization was abnormal compared to NH listeners. Future CI signal processing should aim to improve the transmission of both F0 cues and VTL cues, as a normal gender categorization may benefit speech understanding in competing talker situations.

show abstract

Section: Introductionmentioning

confidence: 97%

Gender Categorization Is Abnormal in Cochlear Implant Users

Fuller¹,

Gaudrain²,

Clarke³

et al. 2014

JARO

Self Cite

112

View full text Add to dashboard Cite

show abstract

“…However, as the signal delivered in each electrode is modulated by the speech envelope that carries temporal F 0 cues, pitch perception remains limitedly possible. Studies on gender categorization, another task that relies on the perception of temporal and spectral cues of a speaker’s voice, confirmed that CI users mostly rely on temporal voice pitch cues, whereas NH listeners can utilize both spectral and temporal voice pitch cues (Fu, Chinchilla, & Galvin, 2004; Fu, Chinchilla, Nogaki, & Galvin, 2005; Fuller, Gaudrain, et al., 2014; Kovacic & Balaban, 2009, 2010; Wilkinson, Abdel-Hamid, Galvin, Jiang, & Fu, 2013). …”

Section: Introductionmentioning

confidence: 99%

Normal-Hearing Listeners’ and Cochlear Implant Users’ Perception of Pitch Cues in Emotional Speech

et al. 2015

View full text Add to dashboard Cite

In cochlear implants (CIs), acoustic speech cues, especially for pitch, are delivered in a degraded form. This study’s aim is to assess whether due to degraded pitch cues, normal-hearing listeners and CI users employ different perceptual strategies to recognize vocal emotions, and, if so, how these differ. Voice actors were recorded pronouncing a nonce word in four different emotions: anger, sadness, joy, and relief. These recordings’ pitch cues were phonetically analyzed. The recordings were used to test 20 normal-hearing listeners’ and 20 CI users’ emotion recognition. In congruence with previous studies, high-arousal emotions had a higher mean pitch, wider pitch range, and more dominant pitches than low-arousal emotions. Regarding pitch, speakers did not differentiate emotions based on valence but on arousal. Normal-hearing listeners outperformed CI users in emotion recognition, even when presented with CI simulated stimuli. However, only normal-hearing listeners recognized one particular actor’s emotions worse than the other actors’. The groups behaved differently when presented with similar input, showing that they had to employ differing strategies. Considering the respective speaker’s deviating pronunciation, it appears that for normal-hearing listeners, mean pitch is a more salient cue than pitch range, whereas CI users are biased toward pitch range cues.

show abstract

“…Some studies set out to examine whether certain forms of speech would affect talker perception. These studies found that CI users were better at picking up on indexical information when presented with neutral speech (compared to whispered speech; Hazrati et al, 2015), speech presented through broadband (compared to presented through the telephone; Horng et al, 2007), and with original stimuli (compared to F0 manipulated speech; Wilkinson et al, 2013). These studies highlight how different real-life situations might impact talker perception for CI users.…”

Section: Task Characteristicsmentioning

confidence: 92%

Recognizing voices through a cochlear implant: A systematic review

Orena¹,

Sem²

2021

Preprint

View full text Add to dashboard Cite

Objective: Some cochlear implant (CI) users report having difficulty accessing indexical information in the speech signal, presumably due to the transformation from acoustic to electric signal in CI devices. The purpose of this paper was to systematically review and evaluate the existing research on talker perception in CI users. Specifically, we reviewed the performance of CI users in talker discrimination, gender identification, and talker recognition tasks in relation to performance by normal-hearing (NH) listeners. We also examined the different factors (such asparticipant, hearing and device characteristics) that might influence talker perception.Design: We completed a systematic search of the literature with select keywords using citation aggregation software to search Google Scholar. We included primary reports that had at least one group of participants with cochlear implants, and had an experimental task that measured talker or voice perception. Each included study was also evaluated for quality of evidence.Results: The initial search resulted in 1239 references, which were first screened for inclusion and then evaluated in full. Thirty-nine studies examining talker identification, talker discrimimnation, and gender discrimination were included in the final review. The majority of studies were focused on adult postlingual cochlear implant users, with a few studies focused on prelingual implant users. As a group, CI users generally performed above chance in talker perception tasks, but performed worse than NH controls. Nonetheless, a subset of CI users reached the same level of performance as NH participants. CI users relied more heavily on fundamental frequency over vocal tract length cues to distinguish talkers compared to NH listeners. Within groups of CI users, there is moderate evidence for a bimodal benefit for talker perception, and there are mixed findings about the effects of hearing experience. Performance in talker discrimination tasks was related to other linguistic tasks, including word recognition. Conclusion: The current review highlights the challenges faced by CI users in tracking and recognizing voices and how they adapt to it. There is clear evidence that CI users can process indexical information, albeit differently and more effortfully than NH listeners. Recent work has begun to describe some of the factors that might ease the challenges of talker perception in CI users, but further high-quality research is needed to disentangle some of the mixed findings. We conclude by suggesting some future avenues of research to optimize real-world speech outcomes.

show abstract

Voice conversion in cochlear implantation

Cited by 8 publications

References 35 publications

Gender Categorization Is Abnormal in Cochlear Implant Users

Gender Categorization Is Abnormal in Cochlear Implant Users

Normal-Hearing Listeners’ and Cochlear Implant Users’ Perception of Pitch Cues in Emotional Speech

Recognizing voices through a cochlear implant: A systematic review

Contact Info

Product

Resources

About