Deep Learning Based Part-of-Speech Tagging for Malayalam Twitter Data (Special Issue: Deep Learning Techniques for Natural Language Processing)

Abstract

The paper addresses the problem of part-of-speech (POS) tagging for Malayalam tweets. The conversational style of posts/tweets/text in social media data poses a challenge in using general POS tagset for tagging the text. For the current work, a tagset was designed that contains 17 coarse tags and 9915 tweets were tagged manually for experiment and evaluation. The tagged data were evaluated using sequential deep learning methods like recurrent neural network (RNN), gated recurrent units (GRU), long short-term memory (LSTM), and bidirectional LSTM (BLSTM). The training of the model was performed on the tagged tweets, at word level and character level. The experiments were evaluated using measures like precision, recall, f1-measure, and accuracy. During the experiment, it was found that the GRU-based deep learning sequential model at word level gave the highest f1-measure of 0.9254; at character-level, the BLSTM-based deep learning sequential model gave the highest f1-measure of 0.8739. To choose the suitable number of hidden states, we varied it as 4, 16, 32, and 64, and performed training for each. It was observed that the increase in hidden states improved the tagger model. This is an initial work to perform Malayalam Twitter data POS tagging using deep learning sequential models.

Part of speech tagging: a systematic review of deep learning and machine learning approaches

“…This section describes the most commonly deployed performance metrics for validating the performance of ML and DL methods for POS tagging. All the evaluation metrics are based on the different metrics used in the Confusion Matrix, which is a confusion matrix providing information about the Actual and Predicted class which are; True Positive (TP)-assigns correct tags to the given words, false positive (FP)-assigns incorrect tags to the given words, false negative (FN)-not assign any tags to given words [14,55,72].…”

Neural machine translation for Sanskrit to Malayalam using morphology and evolutionary word sense disambiguation

“…A challenge is the disambiguation of word senses by the machine. A wide range of language processing systems is pre-processing using prosedur operasional standar (POS) tagging [17], which greatly increases their accuracy and recall. Applications to speech recognition and analysis, data gathering, and some other NLP tasks are mostly found in the POS tagged (annotated) language corpora.…”

EmployingWikipedia as a Resource for Named Entity Recognition in Morphologically Complex Under-Resourced Languages

“…It has featured in a limited number of NLP tasks, including morphological analysis (Bhavukam et al, 2018), POS tagging (Akhil et al, 2020) and NER (Ajees and Idicula, 2018). However, many studies use small locally generated data sets (Nambiar et al, 2019) or domain specific data sets (Kumar et al, 2019), (Devi et al, 2016), which usually are not freely available.…”