A Study On Incorporating Whisper For Robust Speech Assessment

Zezario, Ryandhimas E.; Chen, Yu-Wen; Fu, Szu-Wei; Tsao, Yu; Wang, Hsin-Min; Fuh, Chiou-Shann

doi:10.1109/icme57554.2024.10688047

2024 IEEE International Conference on Multimedia and Expo (ICME) 2024

DOI: 10.1109/icme57554.2024.10688047

|View full text |Cite

A Study On Incorporating Whisper For Robust Speech Assessment

Ryandhimas E. Zezario,

Yu-Wen Chen,

Szu-Wei Fu

et al.

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

Publication Types

Select...

Article2

Relationship

Self Cite0

Independent2

Authors

Journals

Cited by 2 publications

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Multi-objective non-intrusive hearing-aid speech assessment model

Chiang,

Fu,

Wang

et al. 2024

The Journal of the Acoustical Society of America

View full text Add to dashboard Cite

Because a reference signal is often unavailable in real-world scenarios, reference-free speech quality and intelligibility assessment models are important for many speech processing applications. Despite a great number of deep-learning models that have been applied to build non-intrusive speech assessment approaches and achieve promising performance, studies focusing on the hearing impaired (HI) subjects are limited. This paper presents HASA-Net+, a multi-objective non-intrusive hearing-aid speech assessment model, building upon our previous work, HASA-Net. HASA-Net+ improves HASA-Net in several ways: (1) inclusivity for both normal-hearing and HI listeners, (2) integration with pre-trained speech foundation models and fine-tuning techniques, (3) expansion of predictive capabilities to cover speech quality and intelligibility in diverse conditions, including noisy, denoised, reverberant, dereverberated, and vocoded speech, thereby evaluating its robustness, and (4) validation of the generalization capability using an out-of-domain dataset.

show abstract

Multi-objective non-intrusive hearing-aid speech assessment model

Chiang,

Fu,

Wang

et al. 2024

The Journal of the Acoustical Society of America

View full text Add to dashboard Cite

show abstract

A review on subjective and objective evaluation of synthetic speech

Cooper,

Huang,

Tsao

et al. 2024

Acoust. Sci. & Tech.

View full text Add to dashboard Cite

Evaluating synthetic speech generated by machines is a complicated process, as it involves judging along multiple dimensions including naturalness, intelligibility, and whether the intended purpose is fulfilled. While subjective listening tests conducted with human participants have been the gold standard for synthetic speech evaluation, its costly process design has also motivated the development of automated objective evaluation protocols. In this review, we first provide a historical view of listening test methodologies, from early in-lab comprehension tests to recent large-scale crowdsourcing mean opinion score (MOS) tests. We then recap the development of automatic measures, ranging from signal-based metrics to model-based approaches that utilize deep neural networks or even the latest self-supervised learning techniques. We also describe the VoiceMOS Challenge series, a scientific event we founded that aims to promote the development of data-driven synthetic speech evaluation. Finally, we provide insights into unsolved issues in this field as well as future prospective. This review is expected to serve as an entry point for early academic researchers to enrich their knowledge in this field, as well as speech synthesis practitioners to catch up on the latest developments.

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

A Study On Incorporating Whisper For Robust Speech Assessment

Cited by 2 publications

References 31 publications

Multi-objective non-intrusive hearing-aid speech assessment model

Multi-objective non-intrusive hearing-aid speech assessment model

A review on subjective and objective evaluation of synthetic speech

Contact Info

Product

Resources

About