Benchmarking performance of machine and deep learning-based methodologies for Urdu text document classification

Abstract

In order to provide benchmark performance for Urdu text document classification, the contribution of this paper is manifold. First, it provides a publicly available benchmark dataset manually tagged against 6 classes. Second, it investigates the performance impact of traditional machine learning-based Urdu text document classification methodologies by embedding 10 filter-based feature selection algorithms which have been widely used for other languages. Third, for the very first time, it assesses the performance of various deep learning-based methodologies for Urdu text document classification. In this regard, for experimentation, we adapt 10 deep learning classification methodologies which have produced best performance figures for English text classification. Fourth, it also investigates the performance impact of transfer learning by utilizing Bidirectional Encoder Representations from Transformers approach for Urdu language. Fifth, it evaluates the integrity of a hybrid approach which combines traditional machine learning-based feature engineering and deep learning-based automated feature engineering. Experimental results show that feature selection approach named as normalized difference measure along with support vector machine outshines state-of-the-art performance on two closed source benchmark datasets CLE Urdu Digest 1000k, and CLE Urdu Digest 1Million with a significant margin of 32% and 13%, respectively. Across all three datasets, normalized difference measure outperforms other filter-based feature selection algorithms as it significantly uplifts the performance of all adopted machine learning, deep learning, and hybrid approaches. The source code and presented dataset are available at Github repository https://github.com/minixain/Urdu-Text-Classification.

Advancing natural language processing (NLP) applications of morphologically rich languages with bidirectional encoder representations from transformers (BERT): an empirical case study for Turkish

“…(2) Text Classification: A recent study for Urdu, i.e. a low-resource language, made use of BERT in comparison to SVM for text categorization problem is given in [62]. Traditionally for SVM classification pipeline, they used preprocessing steps such as cleaning, stop-word removal, tokenization and lemmatization together with ten feature filtering algorithms.…”

UTSA: Urdu Text Sentiment Analysis Using Deep Learning Methods

“…In [27], the authors used a single layer CNN with multiple filters for document-level text classification and found it superior to the baseline methods. Asim et al [29] assessed performance of state-of-the-art ML, DL, and hybrid model for document classification. Their experiments showed that the normalized difference measure-based feature selection approach improved the performance of all models.…”

A Comprehensive Evaluation of Metadata-Based Features to Classify Research Paper’s Topics

“…Moreover, they observed that the classification rate exceeds 90% when using more than 4000 features. Some of the recent techniques [29] [30] employed different deep learning models to conduct the document classification and yielded good results. But these techniques are time-consuming.…”