Anuj@DPIL-FIRE2016: A Novel Paraphrase Detection Method in Hindi Language Using Machine Learning

Saini, Anuj; Verma, Aayushi

doi:10.1007/978-3-319-73606-8_11

Cited by 3 publications

(3 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This clearly indicate that the features to be used in detecting the paraphrases are dependent on the language characteristics. Similarly the three teams JU_NLP [15], KS_JU [17] and NLP_NITMZ [17] which submitted their results for all the four languages had showed the difference in accuracy across the family of languages for their lexico-syntactic features. In general, the accuracy of Tamil and Malayalam are lesser when compared to the accuracy obtained by the Hindi and Punjabi languages for the similar lexico-syntactic features.…”

Section: Related Workmentioning

confidence: 89%

“…Another team Anuj [16] submitted his result only for Hindi language without using any similarity measures but by using only the lexical features such as lemma, stop word removal and synonym, yielded the better result than the team which used the similarity-based features. This clearly highlights that the similarity features, lexical or semantic features used to detect the phraphrases are dependent on the natural languge characteristics.…”

Section: Related Workmentioning

confidence: 99%

“…To detect the Tamil paraphrases using encoder-decoder model we have adopted the Neural Machine Translation architecture [11] which was based on the sequence-to-sequence learning [3][19]. The sequence-to-sequence(seq-2-seq) models were pioneered by Ilya Sutskever [15], applied that model for machine translation (MT) task. This is the most promising model in natural language processing (NLP) tasks such as sequence labeling [4][20], machine translation [11], syntactic parsing [21][8], semantic parsing [5].…”

Section: Encoder-decoder Modelmentioning

confidence: 99%

See 2 more Smart Citations

Tamil Paraphrase Detection Using Encoder-Decoder Neural Networks

Kumar¹,

Thenmozhi²,

Subramanian³

2020

IFIP Advances in Information and Communication Technology

View full text Add to dashboard Cite

Detecting paraphrases in Indian languages require critical anlysis on the lexical, syntactic and semantic features. Since the structure of Indian languages differ from the other languages like English, the usage of lexico-syntactic features vary between the Indian languages and plays a critical role in determining the performance of the system. Instead of using various lexico-syntactic similarity features, we aim to apply a complete end-to-end system using deep learning networks with no lexicosyntactic features. In this paper we exploited the encoder-decoder model of deep neural network to analyze the paraphrase sentences in Tamil language and to classify. In this encoder-decoder model, LSTM, GRU units and gNMT are used as layers along with attention mechanism. Using this end-to-end model, there is an increase in f1-measure by 0.5% for the subtask-1 when compared to the state-of-the-art systems. The system was trained and evaluated on DPIL@FIRE2016 Shared Task dataset. To our knowledge, ours is the first deep learning model which validates the training instances of both the subtask-1 and subtask-2 dataset of DPIL shared task.

show abstract

Section: Related Workmentioning

confidence: 89%

Section: Related Workmentioning

confidence: 99%

Section: Encoder-decoder Modelmentioning

confidence: 99%

See 1 more Smart Citation

Tamil Paraphrase Detection Using Encoder-Decoder Neural Networks

Kumar¹,

Thenmozhi²,

Subramanian³

2020

IFIP Advances in Information and Communication Technology

View full text Add to dashboard Cite

show abstract

Sentence similarity detection in Malayalam language using cosine similarity

Gokul

Akhil

Shiva³

2017

2017 2nd IEEE International Conference on Recent Trends in Electronics, Information &Amp; Communication Technology (RTEICT)

View full text Add to dashboard Cite

Creating Paraphrase Identification Corpus for Indian Languages

Kumar

Singh²,

Ramanan³

et al. 2020

Handbook of Research on Emerging Trends and Applications of Machine Learning

View full text Add to dashboard Cite

In recent times, paraphrase identification task has got the attention of the research community. The paraphrase is a phrase or sentence that conveys the same information but using different words or syntactic structure. The Microsoft Research Paraphrase Corpus (MSRP) is a well-known openly available paraphrase corpus of the English language. There is no such publicly available paraphrase corpus for any Indian language (as of now). This chapter explains the creation of paraphrase corpus for Hindi, Tamil, Malayalam, and Punjabi languages. This is the first publicly available corpus for any Indian language. It was used in the shared task on detecting paraphrases for Indian languages (DPIL) held in conjunction with Forum for Information Retrieval & Evaluation (FIRE) 2016. The annotation process was performed by a postgraduate student followed by a two-step proofreading by a linguist and a language expert.

show abstract

Anuj@DPIL-FIRE2016: A Novel Paraphrase Detection Method in Hindi Language Using Machine Learning

Cited by 3 publications

References 7 publications

Tamil Paraphrase Detection Using Encoder-Decoder Neural Networks

Tamil Paraphrase Detection Using Encoder-Decoder Neural Networks

Sentence similarity detection in Malayalam language using cosine similarity

Creating Paraphrase Identification Corpus for Indian Languages

Contact Info

Product

Resources

About