No abstract

This paper summarizes a speech recognition initiative for African languages. More precisely, we propose innovative approaches that address the low-resource property of these languages. For both monolingual and multilingual systems, our methods rely on self-supervised pre-trained models for multiple languages. We tested our method on seven African languages and dialects: Amharic, Darija, Fongbe, Sudanese, Swahili, Wolof, and Yoruba. We first trained monolingual models that were used as baselines, and then proposed proof-of-concepts for systems that handle multiple languages. Our multilingual systems were based on three scenarios: (a) we trained a single model by concate-nating the multilingual corpora; (b) we discussed this first model by testing another joint model that predicts the spoken language using language-specific tokens before the text transcription; and (c) we fed a one-hot encoder vector to the latent feature extractions before training the single model and for inference. For this purpose, a language identification model is required. We also investigated the impact of lexical ambiguity by removing diacritics from text in some languages.

No abstract

The performance of existing transformer-based language models in providing state-of-the-art results on many downstream tasks is well established. However, these models tend to be limited to high-resource languages or are multilingual in nature. The availability of models dedicated to Arabic dialects is limited, and even those that exist primarily support dialects written in Arabic script. This study presents the first BERT models for Moroccan Arabic dialect, also known as Darija, called DarijaBERT, DarijaBERT-arabizi, and DarijaBERT-mix. These models are trained on the largest Arabic monodialectal corpus, supporting both Arabic and Latin character representations of the Moroccan dialect. The models' performance is evaluated and compared to existing multidialectal and multilingual models on four distinct downstream tasks, demonstrating state-of-the-art results. The data collection methodology and pre-training process are described, and the Moroccan Topic Classification Dataset (MTCD) is introduced as the first dataset for topic classification in the Moroccan Arabic dialect. The pre-trained models and MTCD dataset are available to the scientific community.