SemEval-2016 Task 2: Interpretable Semantic Textual Similarity

Agirre, Eneko; González-Agirre, Aitor; López-Gazpio, Iñigo; Maritxalar, Montse; Rigau, Germán; Uria, Larraitz

doi:10.18653/v1/s16-1082

Cited by 95 publications

(105 citation statements)

References 13 publications

Supporting

Mentioning

105

Contrasting

Order By: Relevance

“…This positive meaning ranges from implicatures, i.e., what is suggested in an utterance even though neither expressed nor strictly implied (Blackburn, 2008), to entailments. Other terms used in the literature include implied meanings (Mitkov, 2005), implied alternatives (Rooth, 1985) and semantically similars (Agirre et al, 2013). We do not strictly fit into any of this terminology, we reveal positive interpretations as intuitively done by humans when reading text.…”

Section: Terminology Scope and Focusmentioning

confidence: 78%

Understanding Negation in Positive Terms Using Syntactic Dependencies

Sarabi

Blanco

2016

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

This paper presents a two-step procedure to extract positive meaning from verbal negation. We first generate potential positive interpretations manipulating syntactic dependencies. Then, we score them according to their likelihood. Manual annotations show that positive interpretations are ubiquitous and intuitive to humans. Experimental results show that dependencies are better suited than semantic roles for this task, and automation is possible.

show abstract

Section: Terminology Scope and Focusmentioning

confidence: 78%

Understanding Negation in Positive Terms Using Syntactic Dependencies

Sarabi

Blanco

2016

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

show abstract

“…We randomly split all dataset files of SemEval-2012-2015(Agirre et al, 2012, 2013, 2015 into ten. We used the preparation of the data from (Baudis et al, 2016).…”

Section: Experiments and Resultsmentioning

confidence: 99%

HCTI at SemEval-2017 Task 1: Use convolutional neural network to evaluate Semantic Textual Similarity

Shao¹

2017

Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)

View full text Add to dashboard Cite

This paper describes our convolutional neural network (CNN) system for the Semantic Textual Similarity (STS) task. We calculated semantic similarity score between two sentences by comparing their semantic vectors. We generated a semantic vector by max pooling over every dimension of all word vectors in a sentence. There are two key design tricks used by our system. One is that we trained a CNN to transfer GloVe word vectors to a more proper form for the STS task before pooling. Another is that we trained a fully-connected neural network (FCNN) to transfer the difference of two semantic vectors to the probability distribution over similarity scores. All hyperparameters were empirically tuned. In spite of the simplicity of our neural network system, we achieved a good accuracy and ranked 3rd on primary track of SemEval 2017.

show abstract

“…The task has been used as a community evaluation exercise, the *SEM 2013 shared task on Semantic Textual Similarity (Agirre et al, 2013b). The exercise attracted 14 system runs from 6 teams.…”

Section: Discussionmentioning

confidence: 99%

“…The systems described in this article participated in the *SEM 2013 shared task (Agirre et al, 2013b). Our baseline system was used as the overall task baseline against which all runs were compared.…”

Section: Performance In Shared Taskmentioning

confidence: 99%

Why are these similar? Investigating item similarity types in a large digital library

González-Agirre

Rigau

Agirre

et al. 2015

Asso for Info Science & Tech

Self Cite

View full text Add to dashboard Cite

We introduce a new problem, identifying the type of relation that holds between a pair of similar items in a digital library. Being able to provide a reason why items are similar has applications to recommendation, personalisation and search.We investigate the problem within the context of Europeana, a large digital library containing items related to cultural heritage. A range of types of similarity in this collection were identified. A set of 1500 pairs of items from the collection were annotated using crowdsourcing. A high inter-tagger agreement (average 71.5 Pearson correlation) was obtained and demonstrates that the task is well defined. We also present several approaches to automatically identifying the type of similarity. The best system applies linear regression and achieves a mean Pearson correlation of 71.3, close to human performance.The problem formulation and data set described here were used in a public evaluation exercise, the *SEM shared task on Semantic Textual Similarity. The task attracted the participation of 6 teams, who submitted 14 system runs. All annotations, evaluation scripts and system runs are freely available 1 .

show abstract

SemEval-2016 Task 2: Interpretable Semantic Textual Similarity

Cited by 95 publications

References 13 publications

Understanding Negation in Positive Terms Using Syntactic Dependencies

Understanding Negation in Positive Terms Using Syntactic Dependencies

HCTI at SemEval-2017 Task 1: Use convolutional neural network to evaluate Semantic Textual Similarity

Why are these similar? Investigating item similarity types in a large digital library

Contact Info

Product

Resources

About