Transformer Architecture-Based Transfer Learning for Politeness Prediction in Conversation

Abstract

Politeness is an essential part of a conversation. Like verbal communication, politeness in textual conversation and social media posts is also stimulating. Therefore, the automatic detection of politeness is a significant and relevant problem. The existing literature generally employs classical machine learning-based models like naive Bayes and Support Vector-based trained models for politeness prediction. This paper exploits the state-of-the-art (SOTA) transformer architecture and transfer learning for respectability prediction. The proposed model employs the strengths of context-incorporating large language models, a feed-forward neural network, and an attention mechanism for representation learning of natural language requests. The trained representation is further classified using a softmax function into polite, impolite, and neutral classes. We evaluate the presented model employing two SOTA pre-trained large language models on two benchmark datasets. Our model outperformed the two SOTA and six baseline models, including two domain-specific transformer-based models using both the BERT and RoBERTa language models. The ablation investigation shows that the exclusion of the feed-forward layer displays the highest impact on the presented model. The analysis reveals the batch size and optimization algorithms as effective parameters affecting the model performance.

Hybrid Majority Voting: Prediction and Classification Model for Obesity

“…By evaluating the performance metrics and accuracy rates of each model, we can gain a better understanding of the advancements and limitations in the field of obesity prediction using machine learning techniques. Moreover, the proposed model can be efficiently used for other applications of deep learning and machine learning other than obesity [30][31][32][33][34][35][36][37][38].…”