Analysis of Contextual and Non-contextual Word Embedding Models for Hindi NER with Web Application for Data Collection

Barua, Aindriya; Thara, S; Premjith, B.

doi:10.1007/978-981-16-0401-0_14

Cited by 13 publications

(10 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly, Lorini et al used context-independent multilingual embeddings for their flood recognition system based on online social media called European Flood Awareness System (EFAS) in [16]. However, context-independent word embedding typically fails to capture relevant information [2]. Torres et al explored crisis-related conversations in a cross-lingual setting [26].…”

Section: Related Workmentioning

confidence: 99%

Graph Neural Network Enhanced Language Models for Efficient Multilingual Text Classification

Ghosh¹,

Maji²,

Desarkar³

2022

Preprint

View full text Add to dashboard Cite

Online social media works as a source of various valuable and actionable information during disasters. These information might be available in multiple languages due to the nature of user generated content. An effective system to automatically identify and categorize these actionable information should be capable to handle multiple languages and under limited supervision. However, existing works mostly focus on English language only with the assumption that sufficient labeled data is available. To overcome these challenges, we propose a multilingual disaster related text classification system which is capable to work undervmonolingual, cross-lingual and multilingual lingual scenarios and under limited supervision. Our end-to-end trainable framework combines the versatility of graph neural networks, by applying over the corpus, with the power of transformer based large language models, over examples, with the help of cross-attention between the two. We evaluate our framework over total nine English, Non-English and monolingual datasets invmonolingual, cross-lingual and multilingual lingual classification scenarios. Our framework outperforms state-of-the-art models in disaster domain and multilingual BERT baseline in terms of Weighted F 1 score. We also show the generalizability of the proposed model under limited supervision.

show abstract

Section: Related Workmentioning

confidence: 99%

Graph Neural Network Enhanced Language Models for Efficient Multilingual Text Classification

Ghosh¹,

Maji²,

Desarkar³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Contextual word embeddings, on the other hand, take the context of each word into account when encoding words in a sentence structure. BERT, Roberta and XLM are popular contextual embedding methods based on the transformer architecture [4,9], which is a recent breakthrough in the field of Natural Language Processing (NLP). The transformer was originally introduced as a means of improving neural machine translations [7,29].…”

Section: Contextual Word Embeddingsmentioning

confidence: 99%

“…DistilRoBERTa and XLM are transformer-based models that support both the fine-tuning and feature-based approaches [4]. As discussed earlier, the finetuning approach involves re-using the entire architecture for downstream tasks.…”

Section: Contextual Word Embeddingsmentioning

confidence: 99%

“…In particular, create a sentence embeddings by averaging individual word embeddings. We use the pre-trained Word2Vec model ("word2vec-google-news-300") from the Python Gensim library 4 , where each word is represented by a 300 dimensional vector. The pre-processing steps include tokenisation, lower-casing, punctuation removal, stop-word removal, lemmatisation as well as the removal of out-ofvocabulary words.…”

Section: Experiments Setupmentioning

confidence: 99%

See 1 more Smart Citation

Financial Forecasting with Word Embeddings Extracted from News: A Preliminary Analysis

Barbaglia

Consoli

2021

Communications in Computer and Information Science

View full text Add to dashboard Cite

News represents a rich source of information about financial agents actions and expectations. We rely on word embedding methods to summarize the daily content of news. We assess the added value of the word embeddings extracted from US news, as a case study, by using different language approaches while forecasting the US S&P500 index by means of DeepAR, an advanced neural forecasting method based on auto-regressive Recurrent Neural Networks operating in a probabilistic setting. Although this is currently on-going work, the obtained preliminary results look promising, suggesting an overall validity of the employed methodology.

show abstract

“…In this paper, we tried to bring about machine learning approaches built upon a novel way of generating word embeddings through various pretrained multilingual BERT models (Barua et al, 2020) for improving the dependency relation prediction. These models are fine-tuned to improve 1 and study the variational changes within the models and improving the performance of dependency parsing for Tamil Language.…”

Section: Introductionmentioning

confidence: 99%

BERT-Based Sequence Labelling Approach for Dependency Parsing in Tamil

Kumar¹,

Maharana²,

Murali³

et al. 2022

Proceedings of the Second Workshop on Speech and Language Technologies for Dravidian Languages

View full text Add to dashboard Cite

Dependency parsing is a method for doing surface-level syntactic analysis on natural language texts. The scarcity of any viable tools for doing these tasks in Dravidian Languages has introduced a new line of research into these topics. This paper focuses on a novel approach that uses word-to-word dependency tagging using BERT models to improve the malt parser performance. We used Tamil, a morphologically rich and free word order language. The individual words are tokenized using BERT models and the dependency relations are recognized using Machine Learning Algorithms. Oversampling algorithms such as SMOTE (Chawla et al., 2002) and ADASYN (He et al., 2008) are used to tackle data imbalance and consequently improve parsing results. The results obtained after oversampling (label accuracy of 69.94% IndicBERT-SVM) are used in the malt parser and this can be accustomed to further highlight that featurebased approaches can be used for such tasks.

show abstract

Analysis of Contextual and Non-contextual Word Embedding Models for Hindi NER with Web Application for Data Collection

Cited by 13 publications

References 11 publications

Graph Neural Network Enhanced Language Models for Efficient Multilingual Text Classification

Graph Neural Network Enhanced Language Models for Efficient Multilingual Text Classification

Financial Forecasting with Word Embeddings Extracted from News: A Preliminary Analysis

BERT-Based Sequence Labelling Approach for Dependency Parsing in Tamil

Contact Info

Product

Resources

About