Back to Basics for Monolingual Alignment: Exploiting Word Similarity                     and Contextual Evidence

Sultan, Arafat; Bethard, Steven; Sumner, Tamara

doi:10.1162/tacl_a_00178

Cited by 96 publications

(31 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is in contrast to SARI, which is designed to evaluate simplifications involving paraphrasing. EASSE re-factors the original SAMSA implementation 2 with some modifications: (1) an internal call to the TUPA parser (Hershcovich et al, 2017), which generates the semantic annotations for each original sentence; (2) a modified version of the monolingual word aligner (Sultan et al, 2014) that is compatible with Python 3, and uses Stanford CoreNLP (Manning et al, 2014) 3 through their official Python interface; and (3) a single function call to get a SAMSA score instead of running a series of scripts.…”

Section: Automatic Corpus-level Metricsmentioning

confidence: 99%

EASSE: Easier Automatic Sentence Simplification Evaluation

Alva-Manchego¹,

Martin²,

Scarton³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

We introduce EASSE, a Python package aiming to facilitate and standardise automatic evaluation and comparison of Sentence Simplification (SS) systems. EASSE provides a single access point to a broad range of evaluation resources: standard automatic metrics for assessing SS outputs (e.g. SARI), wordlevel accuracy scores for certain simplification transformations, reference-independent quality estimation features (e.g. compression ratio), and standard test data for SS evaluation (e.g. TurkCorpus). Finally, EASSE generates easy-to-visualise reports on the various metrics and features above and on how a particular SS output fares against reference simplifications. Through experiments, we show that these functionalities allow for better comparison and understanding of the performance of SS systems.

show abstract

Section: Automatic Corpus-level Metricsmentioning

confidence: 99%

EASSE: Easier Automatic Sentence Simplification Evaluation

Alva-Manchego¹,

Martin²,

Scarton³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

show abstract

“…These systems correlate about 80% with similarity scores annotated by humans (Cer et al, 2017). Most systems integrate combinations of multiple algorithms providing partial scores from a number of aspects of the sentences (Sultan et al, 2014). For instance, a typical similarity score between sentences of a pair can be obtained as follows (Pilehvar and Navigli, 2015;Brychcın and Svoboda, 2016):…”

Section: Sts and The Distinction Between Sts Systems And Sentence Repmentioning

confidence: 99%

Unsupervised sentence representations as word information series: Revisiting TF–IDF

Arroyo-Fernández

Méndez-Cruz

Sierra

et al. 2019

Computer Speech & Language

View full text Add to dashboard Cite

Sentence representation at the semantic level is a challenging task for Natural Language Processing and Artificial Intelligence. Despite the advances in word embeddings (i.e. word vector representations), capturing sentence meaning is an open question due to complexities of semantic interactions among words. In this paper, we present an embedding method, which is aimed at learning unsupervised sentence representations from unlabeled text. We propose an unsupervised method that models a sentence as a weighted series of word embeddings. The weights of the word embeddings are fitted by using Shannon's word entropies provided by the Term Frequency-Inverse Document Frequency (TF-IDF) transform. The hyperparameters of the model can be selected according to the properties of data (e.g. sentence length and textual gender). Hyperparameter selection involves word embedding methods and dimensionalities, as well as weighting schemata. Our method offers advantages over existing methods: identifiable modules, short-term training, online inference of (unseen) sentence representations, as well as independence from domain, external knowledge and language resources. Results showed that our model outperformed the state of the art in well-known Semantic Textual Similarity (STS) benchmarks. Moreover, our model reached state-of-the-art performance when compared to supervised and knowledge-based STS systems.

show abstract

“…There is much less research related to the measurement of similarity between sentences or short text fragments (Islam and Inkpen, 2008). In order to evaluate the degree of two English sentences semantic similarity, Sultan et al exploited an unsupervised system that relied on word alignment (Sultan et al, 2014) or combined a vector similarity feature with alignment-based similarity (Sultan et al, 2015). Now quite a few researchers apply align words algorithms in order to compute the semantic similarity between two sentences.…”

Section: Related Workmentioning

confidence: 99%

RELATIONS - Workshop on meaning relations between phrases and sentences

Kovatchev¹,

Gold²,

Zesch³

2019

View full text Add to dashboard Cite

Commonsense knowledge relations are crucial for advanced NLU tasks. We examine the learnability of such relations as represented in CONCEPTNET, taking into account their specific properties, which can make relation classification difficult: a given concept pair can be linked by multiple relation types, and relations can have multi-word arguments of diverse semantic types. We explore a neural open world multi-label classification approach that focuses on the evaluation of classification accuracy for individual relations. Based on an in-depth study of the specific properties of the CONCEPTNET resource, we investigate the impact of different relation representations and model variations. Our analysis reveals that the complexity of argument types and relation ambiguity are the most important challenges to address. We design a customized evaluation method to address the incompleteness of the resource that can be expanded in future work.

show abstract

Back to Basics for Monolingual Alignment: Exploiting Word Similarity and Contextual Evidence

Cited by 96 publications

References 11 publications

EASSE: Easier Automatic Sentence Simplification Evaluation

EASSE: Easier Automatic Sentence Simplification Evaluation

Unsupervised sentence representations as word information series: Revisiting TF–IDF

RELATIONS - Workshop on meaning relations between phrases and sentences

Contact Info

Product

Resources

About