A cross-language study of speech recognition systems for English, German, and Hebrew

Abstract

Despite the growing importance of Automatic Speech Recognition (ASR), its application is still challenging, limited, language-dependent, and requires considerable resources. The resources required for ASR are not only technical, they also need to reflect technological trends and cultural diversity. The purpose of this research is to explore ASR performance gaps by a comparative study of American English, German, and Hebrew. Apart from different languages, we also investigate different speaking styles – utterances from spontaneous dialogues and utterances from frontal lectures (TED-like genre). The analysis includes a comparison of the performance of four ASR engines (Google Cloud, Google Search, IBM Watson, and WIT.ai) using four commonly used metrics: Word Error Rate (WER); Character Error Rate (CER); Word Information Lost (WIL); and Match Error Rate (MER). As expected, findings suggest that English ASR systems provide the best results. Contrary to our hypothesis regarding ASR’s low performance for under-resourced languages, we found that the Hebrew and German ASR systems have similar performance. Overall, our findings suggest that ASR performance is language-dependent and system-dependent. Furthermore, ASR may be genre-sensitive, as our results showed for German. This research contributes a valuable insight for improving ubiquitous global consumption and management of knowledge and calls for corporate social responsibility of commercial companies, to develop ASR under Fair, Reasonable, and Non-Discriminatory (FRAND) terms

ArabAlg: A new Dataset for Arabic Speech Command Recognition for Machine Learning Applications

“…• Performance metrics: The accuracy of a system in recognizing spoken words is a pivotal factor that directly influences its utility for a specific application. In ASR field, various metrics are taken into consideration, including the Word Error Rate (WER) [24], the Character Error Rate (CER) [3], the Match Error Rate (MER) [25], and the Phoneme Error Rate (PER) [26], [25]. For instance, the MER reflects the percentage of inaccurately predicted and inserted words.…”