A Graph-Based Approach to String Regeneration

Horvat, Matic; Byrne, William

doi:10.3115/v1/e14-3010

“…For the tasks, we experimented with two different inference procedures. Horvat and Byrne (2014) proposed to solve the task of linearization as that of solving the Asymmetric Traveling Salesman Problem (ATSP) over a graph. We incorporate the same as our inference procedure in the ATSP-EBM-F configuration.…”

Section: Ebm-based Configurations (Beam-ebm-f/atsp-ebm-f)mentioning

confidence: 99%

A Graph-Based Framework for Structured Prediction Tasks in Sanskrit

Krishna

¹

,

Santra

²

,

Gupta

³

et al. 2021

Computational Linguistics

View full text Add to dashboard Cite

We propose a framework using Energy Based Models for multiple structured prediction tasks in Sanskrit. Ours is an arc-factored model, similar to the graph based parsing approaches, and we consider the tasks of word-segmentation, morphological parsing, dependency parsing, syntactic linearisation and prosodification, a prosody level task we introduce in this work. Ours is a search based structured prediction framework, which expects a graph as input, where relevant linguistic information is encoded in the nodes, and the edges are then used to indicate the association between these nodes. Typically the state of the art models for morphosyntactic tasks in morphologically rich languages still rely on hand-crafted features for their performance. But here, we automate the learning of the feature function. The feature function so learnt along with the search space we construct, encode relevant linguistic information for the tasks we consider. This enables us to substantially reduce the training data requirements to as low as 10 % as compared to the data requirements for the neural state of the art models. Our experiments in Czech and Sanskrit show the language agnostic nature of the framework, where we train highly competitive models for both the languages. Moreover, our framework enables to incorporate language specific constraints to prune the search space and to filter the candidates during inference. We obtain significant improvements in morphosyntactic tasks for Sanskrit by incorporating language specific constraints into the model. In all the tasks we discuss for Sanskrit, we either achieve state of the art results or ours is the only data driven solution for those tasks.

show abstract

“…This problem has been studied using both treelike and autoregressive models (de Gispert et al, 2014;Zhang and Clark, 2011;Song et al, 2018). Horvat and Byrne (2014) reduce linearization under an 𝑛-gram model to generalized traveling salesman problems (TSP), but stop short of extending TSP algorithms to neural models, as we do in this work.…”

Section: Related Workmentioning

confidence: 99%

Studying word order through iterative shuffling

Malkin¹,

Lanka²,

Goel³

et al. 2021

Preprint

View full text Add to dashboard Cite

As neural language models approach human performance on NLP benchmark tasks, their advances are widely seen as evidence of an increasingly complex understanding of syntax. This view rests upon a hypothesis that has not yet been empirically tested: that word order encodes meaning essential to performing these tasks. We refute this hypothesis in many cases: in the GLUE suite and in various genres of English text, the words in a sentence or phrase can rarely be permuted to form a phrase carrying substantially different information. Our surprising result relies on inference by iterative shuffling (IBIS), a novel, efficient procedure that finds the ordering of a bag of words having the highest likelihood under a fixed language model. IBIS can use any black-box model without additional training and is superior to existing word ordering algorithms. Coalescing our findings, we discuss how shuffling inference procedures such as IBIS can benefit language modeling and constrained generation.

show abstract

“…This problem has been studied using both treelike and autoregressive models (de Gispert et al, 2014;Zhang and Clark, 2011;Song et al, 2018). Horvat and Byrne (2014) reduce linearization under an -gram model to generalized traveling salesman problems (TSP), but stop short of extending TSP algorithms to neural models, as we do in this work.…”

Section: Related Workmentioning

confidence: 99%

Studying word order through iterative shuffling

Malkin¹,

Lanka²,

Goel³

et al. 2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

As neural language models approach human performance on NLP benchmark tasks, their advances are widely seen as evidence of an increasingly complex understanding of syntax. This view rests upon a hypothesis that has not yet been empirically tested: that word order encodes meaning essential to performing these tasks. We refute this hypothesis in many cases: in the GLUE suite and in various genres of English text, the words in a sentence or phrase can rarely be permuted to form a phrase carrying substantially different information. Our surprising result relies on inference by iterative shuffling (IBIS), a novel, efficient procedure that finds the ordering of a bag of words having the highest likelihood under a fixed language model. IBIS can use any black-box model without additional training and is superior to existing word ordering algorithms. Coalescing our findings, we discuss how shuffling inference procedures such as IBIS can benefit language modeling and constrained generation.

show abstract

A Graph-Based Approach to String Regeneration

Cited by 9 publications

References 32 publications

A Graph-Based Framework for Structured Prediction Tasks in Sanskrit

A Graph-Based Framework for Structured Prediction Tasks in Sanskrit

Studying word order through iterative shuffling

Studying word order through iterative shuffling

Contact Info

Product

Resources

About