An in-depth exploration of Bangla blog post classification

Islam, Tanvirul; Prince, Ashik Iqbal; Khan, Md. Mehedee Zaman; Jabiullah, Md. Ismail; Habib, Md. Tarek

doi:10.11591/eei.v10i2.2873

Cited by 6 publications

(4 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this method, for keyword matching we used a predefined list of vulgar words with automatically filtered-out words that had no probability of occurrence in vulgar context, as explained in Section 3.3. The method achieved accuracies of 0.2, 0.245, 0.3, 0.324, 0.363, 0.385, 0.427, 0.449, 0.467, and 0.475 within the top 10,20,30,40,50,60,70,80,90, and 100 extracted words, respectively (see Table 4 and Figure 9). Moreover, for the longer word lists, the method, despite filtering out on average only half of the actually non-vulgar words, achieved accuracies close to purely human-based filtering.…”

Section: Baseline 2: Keyword-matching Methods Based On Tf-idf Term Ex...mentioning

confidence: 97%

See 1 more Smart Citation

Automatic Vulgar Word Extraction Method with Application to Vulgar Remark Detection in Chittagonian Dialect of Bangla

Mahmud,

Ptaszynski,

Masui

2023

Applied Sciences

View full text Add to dashboard Cite

The proliferation of the internet, especially on social media platforms, has amplified the prevalence of cyberbullying and harassment. Addressing this issue involves harnessing natural language processing (NLP) and machine learning (ML) techniques for the automatic detection of harmful content. However, these methods encounter challenges when applied to low-resource languages like the Chittagonian dialect of Bangla. This study compares two approaches for identifying offensive language containing vulgar remarks in Chittagonian. The first relies on basic keyword matching, while the second employs machine learning and deep learning techniques. The keyword-matching approach involves scanning the text for vulgar words using a predefined lexicon. Despite its simplicity, this method establishes a strong foundation for more sophisticated ML and deep learning approaches. An issue with this approach is the need for constant updates to the lexicon. To address this, we propose an automatic method for extracting vulgar words from linguistic data, achieving near-human performance and ensuring adaptability to evolving vulgar language. Insights from the keyword-matching method inform the optimization of machine learning and deep learning-based techniques. These methods initially train models to identify vulgar context using patterns and linguistic features from labeled datasets. Our dataset, comprising social media posts, comments, and forum discussions from Facebook, is thoroughly detailed for future reference in similar studies. The results indicate that while keyword matching provides reasonable results, it struggles to capture nuanced variations and phrases in specific vulgar contexts, rendering it less robust for practical use. This contradicts the assumption that vulgarity solely relies on specific vulgar words. In contrast, methods based on deep learning and machine learning excel in identifying deeper linguistic patterns. Comparing SimpleRNN models using Word2Vec and fastText embeddings, which achieved accuracies ranging from 0.84 to 0.90, logistic regression (LR) demonstrated remarkable accuracy at 0.91. This highlights a common issue with neural network-based algorithms, namely, that they typically require larger datasets for adequate generalization and competitive performance compared to conventional approaches like LR.

show abstract

Section: Baseline 2: Keyword-matching Methods Based On Tf-idf Term Ex...mentioning

confidence: 97%

“…MNB performs well when using features that represent word counts or term frequencies in the context of text classification. Despite its simplicity, MNB frequently performs surprisingly well in tasks like sentiment analysis and document categorization [80].…”

Section: Multinomial Naive Bayesmentioning

confidence: 99%

Automatic Vulgar Word Extraction Method with Application to Vulgar Remark Detection in Chittagonian Dialect of Bangla

Mahmud,

Ptaszynski,

Masui

2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…In Table 7, the sentences are equivalent to each other in words and meanings. The BLEU score of this particular example is measured at 0.67 using a 1 gram probability weight [24]. 1 gram BLEU score is 1 when at least one reference sentence equals the candidate sentence in tokens (Table 4 where output sentences are already meaningful candidates.)…”

Section: Bidirectional Translatormentioning

confidence: 99%

Inclusive bidirectional conversion system between Chittagonian and standard Bangla

Hossain

Milon²,

Sabbir

et al. 2022

Bulletin EEI

View full text Add to dashboard Cite

In recent years, the Bangla language has come out as a very prominent figure in the world of natural language processing. Many researchers have produced exemplary works on the language, but there has not been any notable work on its highly enriched dialects due to the lack of resources and the diversity and complexity in its grammatical structure. This paper has suggested a bidirectional conversion system for one of the most widely used dialects of Bangla; the Chittagonian dialect. Our method employs a bidirectional lexicon that uses binary search, word-to-word mapping, and morphological transformations. The system has achieved an accuracy rate of 95.86% for Chittagonian to standard Bangla and 93.89% in the case of standard Bangla to Chittagonian. We have also provided an efficient word suggestion module, and it has yielded satisfactory results.

show abstract

“…The data are labeled into three, i.e., abusive, not abusive and abusive but not offensive. Islam et al introduced a bangla blog article classification system in [17]. Abusive language detection of Twitter text content [18] is implemented utilizing the bidirectional recurrent neural network (BiRNN) method.…”

Section: Introductionmentioning

confidence: 99%

An evolutionary approach to comparative analysis of detecting Bangla abusive text

2021

Self Cite

View full text Add to dashboard Cite

The use of Bangla abusive texts has been accelerated with the progressive use of social media. Through this platform, one can spread the hatred or negativity in a viral form. Plenty of research has been done on detecting abusive text in the English language. Bangla abusive text detection has not been done to a great extent. In this experimental study, we have applied three distinct approaches to a comprehensive dataset to obtain a better outcome. In the first study, a large dataset collected from Facebook and YouTube has been utilized to detect abusive texts. After extensive pre-processing and feature extraction, a set of consciously selected supervised machine learning classifiers i.e. multinomial Naïve Bayes (MNB), multi layer perceptron (MLP), support vector machine (SVM), decision tree, random forrest, stochastic gradient descent (SGD), ridge, perceptron and k-nearest neighbors (k-NN) has been applied to determine the best result. The second experiment is conducted by constructing a balanced dataset by random under sampling the majority class and finally, a Bengali stemmer is employed on the dataset and then the final experiment is conducted. In all three experiments, SVM with the full dataset obtained the highest accuracy of 88%.

show abstract

An in-depth exploration of Bangla blog post classification

Cited by 6 publications

References 18 publications

Automatic Vulgar Word Extraction Method with Application to Vulgar Remark Detection in Chittagonian Dialect of Bangla

Automatic Vulgar Word Extraction Method with Application to Vulgar Remark Detection in Chittagonian Dialect of Bangla

Inclusive bidirectional conversion system between Chittagonian and standard Bangla

An evolutionary approach to comparative analysis of detecting Bangla abusive text

Contact Info

Product

Resources

About