Code-Switching in Automatic Speech Recognition: The Issues and Future Directions

Abstract

Code-switching (CS) in spoken language is where the speech has two or more languages within an utterance. It is an unsolved issue in automatic speech recognition (ASR) research as ASR needs to recognise speech in bilingual and multilingual settings, where the accuracy of ASR systems declines with CS due to pronunciation variation. There are very few reviews carried out on CS, with none conducted on bilingual and multilingual CS ASR systems. This study investigates the importance of CS in bilingual and multilingual speech recognition systems. To meet the objective of this study, two research questions were formulated, which cover both the current issues and the direction of the research. Our review focuses on databases, acoustic and language modelling, and evaluation metrics. Using selected keywords, this research has identified 274 papers and selected 42 experimental papers for review, of which 24 (representing 57%) have discussed CS, while the rest look at multilingual ASR research. The selected papers cover many well-resourced and under-resourced languages, and novel techniques to manage CS in ASR systems, which are mapping, combining and merging the phone sets of the languages experimented with in the research. Our review also examines the performance of those methods. This review found a significant variation in the performance of CS in terms of word error rates, indicating an inconsistency in the ability of ASRs to handle CS. In the conclusion, we suggest several future directions that address the issues identified in this review.

Smart Wheelchair Controlled Through a Vision-Based Autonomous System

“…Prior to the implementation of our project, we went through the process of researching several types of wheelchairs ( [3]- [8], [14], [15], [18], [19], [21], [41], [42], [57]). A wheelchair that is manually operated is a device that allows the user to sit in it and move around freely.…”

“…3) The primary advantage of our wheelchair is that it can provide greater independence to users with limited mobility. With traditional joystick-controlled wheelchairs, users must have sufficient upper body strength and dexterity to operate the joystick effectively [12]- [15]. However, with a vision-based autonomous wheelchair, users can navigate the environment with minimal physical effort [18], [21], [31].…”

“…One of the most significant benefits of our wheelchair is that it can help users who have restricted mobility achieve greater levels of independence. It is necessary for users of traditional wheelchairs that are controlled by joysticks to possess adequate upper-body strength and dexterity in order to operate the joystick successfully [12][13][14][15]. Nevertheless, users of a vision-based autonomous wheelchair are able to navigate the world with a minimal amount of physical exertion [36], [40], [49].…”

“…The only way for a person to get around here is rely on a person (nurse) to take care of the wheelchair during their movement. Another choice is to get a wheelchair which is mounted on a joystick [2], [3], thereby controlling the movement of the wheelchair. However, a joystick-based wheelchair is ineffective in case of complete paralysis [4].…”

Distributed Communicative Language Training Platform Using Automatic Speech Recognition Technology for Smart University

“…The system listens to the voice and determines the words it hears. ASR consists of deep learning and natural language processing (NLP) technologies as its core components [8].…”