A Comprehensive Survey of Techniques Used for Part-of-Speech Tagging of Code-Mixed Social Media Text

Abstract

Part-of-speech tagging faces unique difficulties when dealing with code-mixed social media text, which combines multiple languages in informal content created by users. In India, many web users employ a mixture of regional languages and English on platforms like Facebook, Instagram, and WhatsApp to express their messages and emotions. Text derived from social media is used in a variety of applications, such as speech recognition, machine learning, information retrieval, question answering, sentiment analysis, and named entity recognition. Due to training with monolingual texts, natural language processing tools such as part-of-speech taggers and parsers don't perform well. Assigning grammatical labels to individual words (such as verbs, adjectives, and nouns) is a critical task in natural language processing. This review paper extensively surveys the existing literature on part-of-speech tagging specifically developed for Indian and Foreign code-mixed social media text. We examine and categorize the approaches utilized in prior studies, taking into account the diverse techniques and methodologies employed to handle the complexities of code-mixed data. These approaches encompass rule-based methods, statistical models, and deep learning techniques such as recurrent neural networks and transformers. To enable comprehensive analysis, we compare and evaluate the performance of various state-of-the-art code-mixed part-of-speech taggers using benchmark datasets. We discuss the evaluation metrics used in these studies. Lastly, we explore the challenges introduced by noisy and informal language commonly found in code-mixed social media text. This review paper serves as a valuable resource for researchers and practitioners seeking to understand the current state of the art in code-mixed part-of-speech tagging for social media text. It offers insights into the strengths and weaknesses of existing approaches, identifies research gaps, and proposes potential avenues for future research to advance the field.