Improving Code-mixed POS Tagging Using Code-mixed Embeddings

Abstract

Social media data has become invaluable component of business analytics. A multitude of nuances of social media text make the job of conventional text analytical tools difficult. Code-mixing of text is a phenomenon prevalent among social media users, wherein words used are borrowed from multiple languages, though written in the commonly understood roman script. All the existing supervised learning methods for tasks such as Parts Of Speech (POS) tagging for code-mixed social media (CMSM) text typically depend on a large amount of training data. Preparation of such large training data is resource-intensive, requiring expertise in multiple languages. Though the preparation of small dataset is possible, the out of vocabulary (OOV) words pose major difficulty, while learning models from CMSM text as the number of different ways of writing non-native words in roman script is huge. POS tagging for code-mixed text is non-trivial, as tagging should deal with syntactic rules of multiple languages. The important research question addressed by this article is whether abundantly available unlabeled data can help in resolving the difficulties posed by code-mixed text for POS tagging. We develop an approach for scraping and building word embeddings for code-mixed text illustrating it for Bengali-English, Hindi-English, and Telugu-English code-mixing scenarios. We used a hierarchical deep recurrent neural network with linear-chain CRF layer on top of it to improve the performance of POS tagging in CMSM text by capturing contextual word features and character-sequence–based information. We prepared a labeled resource for POS tagging of CMSM text by correcting 19% of labels from an existing resource. A detailed analysis of the performance of our approach with varying levels of code-mixing is provided. The results indicate that the F1-score of our approach with custom embeddings is better than the CRF-based baseline by 5.81%, 5.69%, and 6.3% in Bengali, Hindi , and Telugu languages, respectively.

Automatic Identification of Light Verb Constructions: A Review

“…In particular, the use of word embeddings with character n-grams to represent out-of-vocabulary (OOV) words (Bojanowski et al, 2017) which are often either intentionally/unintentionally misspelt words or slang words that are prevalent in social media can potentially help as they are able to reflect the semantic relationships between words. Two interesting ideas to consider is to train word embeddings using a code-mix corpus as was done for POS tagging by Bhattu et al (2020), and to include LVCs and other MWEs as single tokens in embeddings.…”

A Pre-trained Transformer and CNN model with Joint Language ID and Part-of-Speech Tagging for Code-Mixed Social-Media Text

“…CM POS tagging: (Bhattu et al, 2020a) addressed the problem of prediction of POS tags for OOV words in low resource languages using character-based word embedding as input features to a Bi-LSTM and CRF network. (Ball and Garrette, 2018) used a meta embedding approach for the part of speech tagging where the word is represented in both code-mixed languages.…”

“…Bi-LSTM CRF (I): This is the most commonly used model for the LI and POS tasks. It uses Bi-LSTM and CRF consecutively for POS tagging the CM text (Aguilar and Solorio, 2019;Bhattu et al, 2020a).…”

LstFcFedLear: A LSTM‐FC with Vertical Federated Learning Network for Fault Prediction

“…This provides an important basis for industry equipment failure prediction [1,2]. In order to further improve the accuracy of various failure predictions, many new technologies (for example, artificial intelligence, big data, and blockchain) have been gradually applied to factories [3,4].…”

“…But so far, the complex algorithm model [7,8] still has many limitations. In order to overcome these shortcomings, some studies have adopted machine learning algorithms, such as neural networks [4] and support vector machines (SVM) [5] to predict failure types. These studies have promoted the development of probabilistic models to a certain extent [9,10], but probabilistic models lack clear physical meaning in fault prediction.…”