BnVec: Towards the Development of Word Embedding for Bangla Language Processing

Kowsher, Md.; Uddin, Md. Jashim; Tahabilder, Anik; Prottasha, Nusrat Jahan; Ahmed, Mahid; Alam, Kazi Masudul; Sultana, Tamanna

doi:10.14419/ijet.v10i2.31538

Cited by 5 publications

(1 citation statement)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We assessed Bangla-BERT on four downstream tasks for Bangla language comprehension and these are cross-lingual sentiment analysis, named entity recognition, binary Text Classification, and multi-class sentiment analysis. In addition, we have evaluated Bangla-BERT to the multilingual variant of BERT, including other enhanced neural techniques such as fasttext [44], word2vec [45] for findings for each task as a baseline.…”

Section: Evaluation and Resultsmentioning

confidence: 99%

Bangla-BERT: Transformer-Based Efficient Model for Transfer Learning and Language Understanding

et al. 2022

Self Cite

View full text Add to dashboard Cite

The advent of pre-trained language models has directed a new era of Natural Language Processing (NLP), enabling us to create powerful language models. Among these models, Transformer-based models like BERT have grown in popularity due to their cutting-edge effectiveness. However, these models heavily rely on resource-intensive languages, forcing other languages into multilingual models(mBERT). The two fundamental challenges with mBERT become significantly more challenging in a resource-constrained language like Bangla. It was trained on a limited and organized dataset and contained weights for all other languages. Besides, current research on other languages suggests that a language-specific BERT model will exceed multilingual ones. This paper introduces Bangla-BERT a , a monolingual BERT model for the Bangla language. Despite the limited data available for NLP tasks in Bangla, we perform pre-training on the largest Bangla language model dataset, BanglaLM, which we constructed using 40 GB of text data. Bangla-BERT achieves the highest results in all datasets and vastly improves the state-of-the-art performance in binary linguistic classification, multilabel extraction, and named entity recognition, outperforming multilingual BERT and other previous research. The pre-trained model is assessed against several non-contextual models such as Bangla fasttext and word2vec the downstream tasks. Finally, this model is evaluated by transfer learning based on hybrid deep learning models such as LSTM, CNN, and CRF in NER, and it is observed that Bangla-BERT outperforms state-of-the-art methods. The proposed Bangla-BERT model is assessed by using benchmark datasets, including Banfakenews, Sentiment Analysis on Bengali News Comments, and Cross-lingual Sentiment Analysis in Bengali. Finally, it is concluded that Bangla-BERT surpasses all prior state-of-the-art results by 3.52%, 2.2%, and 5.3%. a https://huggingface.co/Kowsher/bangla-bert INDEX TERMS Bangla NLP, BERT-Base, Large Corpus,Transformer I. INTRODUCTION P RE-TRAINED language models based on the transformer architecture have become an absolute standard for state-of-the-art performance on a wide variety of natural language processing applications [1].BERT, a renowned transformer-based technique, brought a great revolution that had huge impacts on the evolution of NLP [2]. Since its release as an academic research paper, this technologically pioneering NLP model has amazed the AI world. It's the firstever deeply bidirectional and fully unsupervised technique

show abstract

Section: Evaluation and Resultsmentioning

confidence: 99%