A Review of Bangla Natural Language Processing Tasks and the Utility of Transformer Models

Abstract

Bangla -ranked as the 6 𝑡ℎ most widely spoken language across the world, 1 with 230 million native speakersis still considered as a low-resource language in the natural language processing (NLP) community. With three decades of research, Bangla NLP (BNLP) is still lagging behind mainly due to the scarcity of resources and the challenges that come with it. There is sparse work in different areas of BNLP; however, a thorough survey reporting previous work and recent advances is yet to be done. In this study, we first provide a review of Bangla NLP tasks, resources, and tools available to the research community; we benchmark datasets collected from various platforms for nine NLP tasks using current state-of-the-art algorithms (i.e., transformer-based models). We provide comparative results for the studied NLP tasks by comparing monolingual vs. multilingual models of varying sizes. We report our results using both individual and consolidated datasets and provide data splits for future research. We reviewed a total of 108 papers and conducted 175 sets of experiments. Our results show promising performance using transformer-based models while highlighting the trade-off with computational costs. We hope that such a comprehensive survey will motivate the community to build on and further advance the research on Bangla NLP.

Z-Index at BLP-2023 Task 2: A Comparative Study on Sentiment Analysis

“…Early research includes rule-based and classical methodologies whereas recent studies include deep learning-based and pretrained language models. Researchers have been trying to develop resources over time and as a result, manual and semi-supervised approaches (Chowdhury and Chowdhury, 2014;Alam et al, 2021;Islam et al, 2021Islam et al, , 2023Kabir et al, 2023) have been adopted in developing sentiment classification datasets. Chowdhury and Chowdhury (2014) used a semi-supervised technique to annotate data and train classical models.…”

“…The interest in low-resource languages is growing over time in sentiment analysis (Batanović et al, 2016;Nabil et al, 2015;Muhammad et al, 2023). Unlike other languages, a limited number † The authors contributed equally to this work of study has been done to develop resources for Bangla sentiment analysis (Hasan et al, 2020a;Alam et al, 2021;Islam et al, 2021;Hasan et al, 2023b;Islam et al, 2023). From the perspective of modeling, there have been studied both classical (i.e., SVM, RF, Naive Bayes) and deep learning (i.e., CNN, LSTM) models.…”

“…From the perspective of modeling, there have been studied both classical (i.e., SVM, RF, Naive Bayes) and deep learning (i.e., CNN, LSTM) models. Pretrained language models (i.e., BERT, XLM-RoBERTa, DistilBERT) have also been studied in recent years (Hasan et al, 2020a;Alam et al, 2021) for sentiment classification. Due to the availability of public data and inadequate information on annotation agreements (Alam et al, 2021), it is challenging for the researchers to focus on this area.…”

“…Pretrained language models (i.e., BERT, XLM-RoBERTa, DistilBERT) have also been studied in recent years (Hasan et al, 2020a;Alam et al, 2021) for sentiment classification. Due to the availability of public data and inadequate information on annotation agreements (Alam et al, 2021), it is challenging for the researchers to focus on this area. This shared task provides a dataset by combining perfect and moderate agreement to shed light on sentiment analysis.…”

Bangla text generation system by incorporating attention in sequence-to-sequence model

“…Some exciting yet typical applications of textual NLP-based tools are text classifier, translator, entity identifier, summarizer, answering question system. Automated text generation providing some source text is a crucial component of many of those NLP systems where it produces coherent human-like text as output [3]. Sequence-to-Sequence, shortly known as Seq2Seq, is an algorithmic model that takes an input sequence, processes it, produces an output sequence, and text generation problems can be modeled using it.…”