Few-Shot Table-to-Text Generation with Prototype Memory

Su, Yixuan; Meng, Zaiqiao; Baker, Simon; Collier, Nigel

doi:10.48550/arxiv.2108.12516

Cited by 4 publications

(1 citation statement)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(Li et al, 2021) uses a prefix set of tokens to better control the topic of the generated text. (Su et al, 2021a) extracts free-form prototypes from a large knowledge base to control the structural formation of the generated text. One of the recent works.…”

Section: Categorymentioning

confidence: 99%

ReTAG: Reasoning Aware Table to Analytic Text Generation

Ghosal,

Nema,

Raghuveer

2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

The task of table summarization involves generating text that both succinctly and accurately represents the table or a specific set of highlighted cells within a table. While significant progress has been made in table to text generation techniques, models still mostly generate descriptive summaries, which reiterates the information contained within the table in sentences. Through analysis of popular table to text benchmarks (ToTTo (Parikh et al., 2020) and InfoTabs (Gupta et al., 2020)) we observe that in order to generate the ideal summary, multiple types of reasoning is needed coupled with access to knowledge beyond the scope of the table. To address this gap, we propose RETAG, a table and reasoning aware model that uses vector-quantization to infuse different types of analytical reasoning into the output. RETAG achieves 2.2%, 2.9% improvement on the PARENT metric in the relevant slice of ToTTo and InfoTabs for the table to text generation task over state of the art baselines. Through human evaluation, we observe that output from RETAG is upto 12% more faithful and analytical compared to a strong table-aware model. To the best of our knowledge, RETAG is the first model that can controllably use multiple reasoning methods within a structure-aware sequence to sequence model to surpass state of the art performance in multiple table to text tasks. We extend (and open source 35.6K analytical, 55.9k descriptive instances) the ToTTo, InfoTabs datasets with the reasoning categories used in each reference sentences.

show abstract

Section: Categorymentioning

confidence: 99%

ReTAG: Reasoning Aware Table to Analytic Text Generation

Ghosal,

Nema,

Raghuveer

2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

show abstract

Plan-then-Generate: Controlled Data-to-Text Generation via Planning

Su¹,

Vandyke²,

Wang³

et al. 2021

Findings of the Association for Computational Linguistics: EMNLP 2021

View full text Add to dashboard Cite

Recent developments in neural networks have led to the advance in data-to-text generation. However, the lack of ability of neural models to control the structure of generated output can be limiting in certain real-world applications. In this study, we propose a novel Plan-then-Generate (PlanGen) framework to improve the controllability of neural data-totext models. Extensive experiments and analyses are conducted on two benchmark datasets, ToTTo and WebNLG. The results show that our model is able to control both the intrasentence and inter-sentence structure of the generated output. Furthermore, empirical comparisons against previous state-of-the-art methods show that our model improves the generation quality as well as the output diversity as judged by human and automatic evaluations.

show abstract

TaCL: Improving BERT Pre-training with Token-aware Contrastive Learning

Liu

Meng

et al. 2022

Findings of the Association for Computational Linguistics: NAACL 2022

View full text Add to dashboard Cite

Masked language models (MLMs) such as BERT have revolutionized the field of Natural Language Understanding in the past few years. However, existing pre-trained MLMs often output an anisotropic distribution of token representations that occupies a narrow subset of the entire representation space. Such token representations are not ideal, especially for tasks that demand discriminative semantic meanings of distinct tokens. In this work, we propose TaCL (Token-aware Contrastive Learning), a novel continual pre-training approach that encourages BERT to learn an isotropic and discriminative distribution of token representations. TaCL is fully unsupervised and requires no additional data. We extensively test our approach on a wide range of English and Chinese benchmarks. The results show that TaCL brings consistent and notable improvements over the original BERT model. Furthermore, we conduct detailed analysis to reveal the merits and inner-workings of our approach. 1 * Work was done prior to joining Amazon. 1 Our code and pre-trained models are publicly available at https://github.com/yxuansu/TaCL

show abstract

Few-Shot Table-to-Text Generation with Prototype Memory

Cited by 4 publications

References 27 publications

ReTAG: Reasoning Aware Table to Analytic Text Generation

ReTAG: Reasoning Aware Table to Analytic Text Generation

Plan-then-Generate: Controlled Data-to-Text Generation via Planning

TaCL: Improving BERT Pre-training with Token-aware Contrastive Learning

Contact Info

Product

Resources

About