An Enhanced Neural Word Embedding Model for Transfer Learning

Abstract

Due to the expansion of data generation, more and more natural language processing (NLP) tasks are needing to be solved. For this, word representation plays a vital role. Computation-based word embedding in various high languages is very useful. However, until now, low-resource languages such as Bangla have had very limited resources available in terms of models, toolkits, and datasets. Considering this fact, in this paper, an enhanced BanglaFastText word embedding model is developed using Python and two large pre-trained Bangla models of FastText (Skip-gram and cbow). These pre-trained models were trained on a collected large Bangla corpus (around 20 million points of text data, in which every paragraph of text is considered as a data point). BanglaFastText outperformed Facebook’s FastText by a significant margin. To evaluate and analyze the performance of these pre-trained models, the proposed work accomplished text classification based on three popular textual Bangla datasets, and developed models using various machine learning classical approaches, as well as a deep neural network. The evaluations showed a superior performance over existing word embedding techniques and the Facebook Bangla FastText pre-trained model for Bangla NLP. In addition, the performance in the original work concerning these textual datasets provides excellent results. A Python toolkit is proposed, which is convenient for accessing the models and using the models for word embedding, obtaining semantic relationships word-by-word or sentence-by-sentence; sentence embedding for classical machine learning approaches; and also the unsupervised finetuning of any Bangla linguistic dataset.

Facial Image Generation from Bangla Textual Description using DCGAN and Bangla FastText

“…The proposed system utilized DCGAN+Bangla Fasttext to generate face images from the corresponding Bangla descriptions. Firstly the Bangla text description is fed to Bangla Fasttext [2] which returns a [300 × 1] shaped text embedding. A random noise vector with a shape of [100 × 1] along with the achieved text embedding is passed to the generator.…”

“…A work [8] from 2021 suggested that using a pre-trained text encoder enhances the performance of text-to-image synthesis. The proposed system employs Bangla FastText [2] as pretrained text encoder. Bangla FastText is trained using 20 million Bangla data.…”

“…The proposed model consists of two primary components, namely a text encoder and an image generator. The utilization of Bangla FastText [2] by the text encoder serves the purpose of encoding Bangla text into a latent vector representation that adeptly captures the semantic information that is intrinsic to the text. Subsequently, the image generator utilizes a Deep Convolutional Generative Adversarial Network (DCGAN) architecture to produce facial images that align with the encoded textual depiction.…”

Detecting Cybercrime: An Evaluation of Machine Learning and Deep Learning Using Natural Language Processing Techniques on the Social Network

“…The importance of word embedding cannot be overstated in approaches based on deep learning architecture because data must be in numerical form. Many word embeddings have been proposed to date, and we evaluated the frameworks proposed using the two most prevalent word embeddings regarded as separate dimensions [30].…”

“…The Word2Vec method employs constructing a vector using continuous bag-of-words (CBOW), the Skip-gram model, and two hidden layers of shallow neural networks with high dimensions for each word; take the next step in our research. Using Skip-grams [30] delves into the corpus of word w and their context. The purpose is to increase the likelihood as much as possible:…”