Multilingual speech recognition in seven languages

438

207

Major progress is being recorded regularly on both the technology and exploitation of automatic speech recognition (ASR) and spoken language systems. However, there are still technological barriers to flexible solutions and user satisfaction under some circumstances. This is related to several factors, such as the sensitivity to the environment (background noise), or the weak representation of grammatical and semantic knowledge.Current research is also emphasizing deficiencies in dealing with variation naturally present in speech. For instance, the lack of robustness to foreign accents precludes the use by specific populations. Also, some applications, like directory assistance, particularly stress the core recognition technology due to the very high active vocabulary (application perplexity). There are actually many factors affecting the speech realization: regional, sociolinguistic, or related to the environment or the speaker herself. These create a wide range of variations that may not be modeled correctly (speaker, gender, speaking rate, vocal effort, regional accent, speaking style, non-stationarity, etc.), especially when resources for system training are scarce. This paper outlines current advances related to these topics.

Section: Pronunciation Modeling Techniquesmentioning

confidence: 99%

Automatic speech recognition and speech variability: A review

BenZeghiba

Mori

Deroo

et al. 2007

438

207

“…Under these conditions, it has not been possible to improve the performance of monolingual speech recognition systems by means of additional data from another language. Kohler [5] and Uebler [6] have come to similar conclusions. We focus on languages spoken within the same country, and hence related at least to some degree by the extensive phonetic and lexical borrowing, sharing, and mixing that takes place in a multilingual society.…”

Section: Introductionmentioning

confidence: 62%

Language-dependent state clustering for multilingual acoustic modelling

Niesler

2007

The development of automatic speech recognition systems requires significant quantities of annotated acoustic data. In South Africa, the large number of spoken languages hampers such data collection efforts. Furthermore, code switching and mixing are commonplace since most citizens speak two or more languages fluently. As a result a considerable degree of phonetic cross pollination between languages can be expected. We investigate whether it is possible to combine speech data from different languages in order to improve the performance of a speech recognition system in any one language. For our investigation we use recently collected Afrikaans, South African English, Xhosa and Zulu speech databases. We extend the decision-tree clustering process normally used to construct tiedstate hidden Markov models to allow the inclusion of languagespecific questions, and compare the performance of systems that allow sharing between languages with those that do not. We find that multilingual acoustic models obtained in this way show a small but consistent improvement over separate-language systems when applied to Afrikaans and English, and to Xhosa and Zulu. The improvement for the latter pair of languages is greater, which is consistent with their larger degree of phonetic similarity.

“…They are now able to provide services such as reservations Lamel et al, 2000;Levin et al, 2000), customer care (Strik et al, 1997) or information retrieval (Zue et al, 2000;Seneff and Polifroni, 2000), 24 h a day 7 days a week. Besides, given the different characteristics and requisites of the final users, the service is expected to be available in several languages (Turunen et al, 2004;Meng et al, 2002;Uebler, 2001) and to provide quick assistance in real time using different modalities such as: voice, gestures, touch screens, Web pages, interactive maps, etc. (Almeida et al, 2002;Wahlster et al, 2001;Gustafson et al, 2000;Oviatt et al, 2000).…”

Section: Motivationmentioning

confidence: 99%

An advanced platform to speed up the design of multilingual dialog applications for multiple modalities

D’Haro

Córdoba

Ferreiros

et al. 2006

In this paper, we present a complete platform for the semiautomatic and simultaneous generation of human-machine dialog applications in two different and separate modalities (Voice and Web) and several languages to provide services oriented to obtaining or modifying the information from a database (data-centered). Given that one of the main objectives of the platform is to unify the application design process regardless of its modality or language and then to complete it with the specific details of each one, the design process begins with a general description of the application, the data model, the database access functions, and a generic finite state diagram consisting of the application flow. With this information, the actions to be carried out in each state of the dialog are defined. Then, the specific characteristics of each modality and language (grammars, prompts, presentation aspects, user levels, etc.) are specified in later assistants. Finally, the scripts that execute the application in the real-time system are automatically generated.We describe each assistant in detail, emphasizing the methodologies followed to ease the design process, especially in its critical aspects. We also describe different strategies and characteristics that we have applied to provide portability, robustness, adaptability and high performance to the platform. We also address important issues in dialog applications such as mixed initiative and over-answering, confirmation handling or providing long lists of information to the user. Finally, the results obtained in a subjective evaluation with different designers and in the creation of two full applications that confirm the usability, flexibility and standardization of the platform, and provide new research directions.