A Novel Deep Auto-Encoder Based Linguistics Clustering Model for Social Text

Abstract

The wide adoption of media and social media has increased the amount of digital content to an enormous level. Natural language processing (NLP) techniques provide an opportunity to extract and explore meaningful information from a large amount of text. Among natural languages, Urdu is one of the widely used languages worldwide for spoken and written communications. Due to its wide adopt-ability, digital content in the Urdu language is increasing briskly, especially with social media and online NEWS feeds. Government agencies and advertisers must filter and understand the content to analyze the trends and cohorts in their interest and national prerogative. Clustering is considered a baseline and one of the first steps in natural language understanding. There are many state-of-the-art clustering techniques specifically for English, French, and Arabic, but no significant research has been conducted in Urdu language processing. Doing it for short text segments is challenging because of limited features and the absence of meaningful language discourse and nuance. Many rule-based NLP techniques are adopted to overcome these issues, relying on human-designed features and rules. Therefore, these methods do not promise remarkable results. Alongside NLP, deep learning techniques are pretty efficient in capturing contextual information with minimal noise compared to other traditional methods. By taking on this challenging job, we develop a deep learning-based technique for Urdu short text clustering for the very first time without a human-designed feature. In this paper, we propose a method of short text clustering using a deep neural network that automatically learns feature representations and clustering assignments simultaneously. This method learns clustering objectives by converting the high dimensional feature space to a low dimensional feature space. Our experiments on the Urdu NEWS headlines dataset show remarkable results compared to state-of-the-art methods.

TAFM: A Recommendation Algorithm Based on Text-Attention Factorization Mechanism

“…Secondly, we need to simplify the representation of the text, which will make our model more advantageous for industrial replication. We will try to further extract textual information by short text clustering [ 43 ], ensemble learning, DistilBERT [ 44 ], and try to optimize the time complexity [ 45 ]. Finally, we would like to incorporate external data (e.g., knowledge graph, speech, images) into CTR prediction to mine the intent behind users with richer features.…”

“…In the past few years, RNNs have achieved great success in speech recognition, language modeling, text translation, and other tasks, but an RNN has the problems of gradient explosion and gradient disappearance due to the presence of continuous cyclic input information. The text component LSTM [ 34 ] introduces a gating unit, which can solve the abovementioned problems faced by RNNs (see Figure 8 ). The mathematical expression of the text component LSTM is shown in Equation ( 11 ): where x t represents the current input value, h t represents the output of the current hidden layer, and h t −1 represents the output of the hidden layer at the previous moment.…”

“…e text component LSTM [34] introduces a gating unit, which can solve the abovementioned problems faced by RNNs (see Figure 8). e mathematical expression of the text component LSTM is shown in Equation (11):…”

Parallel Big Bang-Big Crunch-LSTM Approach for Developing a Marathi Speech Recognition System

“…e proposed DNN-based model outperforms other machine learning approaches. Akram et al [14] projected a linguistic prototype for social text based on deep autoencoder. ey have implemented this model for low resource language Urdu.…”

Cross-Corpus Speech Emotion Recognition Based on Transfer Learning and Multi-Loss Dynamic Adjustment

“…Early research was carried out based on traditional machine learning methods, such as support vector machines (SVM) [ 2 ], Gaussian mixture models (GMM) [ 3 ], hidden Markov models (HMM) [ 4 ], K-Nearest Neighbor (KNN) [ 5 ], and other methods for processing speech features, and in recent years, with the development of deep learning, convolutional neural networks (CNNs) [ 6 ], recurrent neural networks (RNNs) [ 7 ], long short-term memory (LSTM) [ 8 ], deep belief networks (DBNs) [ 9 ], auto-encoders (AEs) [ 10 ], and other methods are applied to speech feature extraction, and these data-driven deep learning methods have obtained excellent performance improvement in SER tasks. Notably, these methods are also widely used in semantic sentiment analysis, with some studies [ 11 , 12 ] using CNNs and AE to learn text feature representations, and these techniques, together with speech sentiment recognition, have driven the development of affective computing research.…”