Evaluation of Automatic Speech Recognition Prototype for Estonian Language in Radiology Domain: A Pilot Study

Paats, Andrus; Alumäe, Tanel; Meister, Einar; Fridolin, Ivo

doi:10.1007/978-3-319-12967-9_26

Cited by 4 publications

(8 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We see that the reports dictated for complicated 3D modalities (CT, MR) have higher WER values than X-ray and ultrasound. The results confirm findings from an earlier study [23] and from our previous results [8], although the relative difference between the modalities is now smaller.…”

Section: Discussionsupporting

confidence: 91%

“…In order to assess the importance and effect of different system development steps, we ran simulations with the test data using earlier system prototype versions. We started with a GMMbased system that was described in [8] and ended with the final system prototype. Table 4 lists average WER results after each improvement.…”

Section: Discussionmentioning

confidence: 99%

“…This paper gives a detailed description of the Estonian radiology ASR system developed by Tallinn University of Technology and North-Estonian Medical Centre (NEMC). The first prototype of the system, using Gaussian Mixture Model (GMM) based acoustic model (AM) was described in [8]. Since then, many improvements have been made to the system, including the integration of domain-adapted deep neural network (DNN) based AMs, language model (LM) adaptation using real dictated texts, smarter handling of sentence breaks and spoken noises in the language model, that have together reduced the word error rate (WER) to around 5%, an almost 75% relative reduction compared to the first prototype.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Implementation of a Radiology Speech Recognition System for Estonian Using Open Source Software

et al. 2017

View full text Add to dashboard Cite

Speech recognition has become increasingly popular in radiology reporting in the last decade. However, developing a speech recognition system for a new language in a highly specific domain requires a lot of resources, expert knowledge and skills. Therefore, commercial vendors do not offer ready-made radiology speech recognition systems for less-resourced languages.This paper describes the implementation of a radiology speech recognition system for Estonian, a language with less than one million native speakers. The system was developed in partnership with a hospital that provided a corpus of written reports for language modeling purposes. Rewrite rules for pre-processing training texts and postprocessing recognition results were created manually based on a small parallel corpus created by the hospital's radiologists, using the Thrax toolkit. Deep neural network based acoustic models were trained based on 216 hours of out-of-domain data and adapted on 14 hours of spoken radiology data, using the Kaldi toolkit. The current word error rate of the system is 5.4%. The system is in active use in real clinical environment.

show abstract

Section: Discussionsupporting

confidence: 91%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Implementation of a Radiology Speech Recognition System for Estonian Using Open Source Software

et al. 2017

View full text Add to dashboard Cite

show abstract

“…The ASR system was utilizing free- and open-source software (Kaldi toolkit, 1 Thrax 2 ) [ 7 , 12 ] based on server-client platform developed in TTÜ 3 , 4 . System components in server side, responsible for converting dictated speech into text, were available for clients over network as reported earlier [ 13 ].…”

Section: Methodsmentioning

confidence: 99%

“…The scientists from Tallinn University of Technology (TTÜ), in collaboration with radiologists from North Estonia Medical Centre (NEMC), Tallinn, Estonia, took a step closer towards an ASR application in radiology for Estonian language by implementing an ASR prototype for Estonian language in radiology domain [ 7 ]. Due to the lack of resources and available commercial ASR system for Estonian language, open-source software components were utilized.…”

Section: Introductionmentioning

confidence: 99%

Retrospective Analysis of Clinical Performance of an Estonian Speech Recognition System for Radiology: Effects of Different Acoustic and Language Models

et al. 2018

Self Cite

View full text Add to dashboard Cite

The aim of this study was to analyze retrospectively the influence of different acoustic and language models in order to determine the most important effects to the clinical performance of an Estonian language-based non-commercial radiology-oriented automatic speech recognition (ASR) system. An ASR system was developed for Estonian language in radiology domain by utilizing open-source software components (Kaldi toolkit, Thrax). The ASR system was trained with the real radiology text reports and dictations collected during development phases. The final version of the ASR system was tested by 11 radiologists who dictated 219 reports in total, in spontaneous manner in a real clinical environment. The audio files collected in the final phase were used to measure the performance of different versions of the ASR system retrospectively. ASR system versions were evaluated by word error rate (WER) for each speaker and modality and by WER difference for the first and the last version of the ASR system. Total average WER for the final version throughout all material was improved from 18.4% of the first version (v1) to 5.8% of the last (v8) version which corresponds to relative improvement of 68.5%. WER improvement was strongly related to modality and radiologist. In summary, the performance of the final ASR system version was close to optimal, delivering similar results to all modalities and being independent on user, the complexity of the radiology reports, user experience, and speech characteristics.

show abstract

Medical Speech Recognition: Reaching Parity with Humans

Edwards

Salloum

Finley

et al. 2017

Speech and Computer

View full text Add to dashboard Cite

Evaluation of Automatic Speech Recognition Prototype for Estonian Language in Radiology Domain: A Pilot Study

Cited by 4 publications

References 0 publications

Implementation of a Radiology Speech Recognition System for Estonian Using Open Source Software

Implementation of a Radiology Speech Recognition System for Estonian Using Open Source Software

Retrospective Analysis of Clinical Performance of an Estonian Speech Recognition System for Radiology: Effects of Different Acoustic and Language Models

Medical Speech Recognition: Reaching Parity with Humans

Contact Info

Product

Resources

About