Few-Shot Table-to-Text Generation with Prototype Memory

Su, Yixuan; Meng, Zaiqiao; Baker, Simon; Collier, Nigel

doi:10.18653/v1/2021.findings-emnlp.77

Cited by 12 publications

(17 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the rising of GPT-2 [60] and BERT [16], the research community has witnessed remarkable progress in the field of language model pre-training on a large amount of free text. Such advancements have led to significant progresses in a wide range of natural language understanding (NLU) tasks [44,87,12,38] and text generation tasks tasks [60,39,61,71,76,75,92,74,77].…”

Section: A Related Workmentioning

confidence: 99%

Language Models Can See: Plugging Visual Controls in Text Generation

Su¹,

Lan²,

Liu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Generative language models (LMs) such as GPT-2/3 can be prompted to generate text with remarkable quality. While they are designed for text-prompted generation, it remains an open question how the generation process could be guided by modalities beyond text such as images. In this work, we propose a training-free framework, called MAGIC (iMAge-Guided text generatIon with CLIP), for plugging in visual controls in the generation process and enabling LMs to perform multimodal tasks (e.g., image captioning) in a zero-shot manner. MAGIC is a simple yet efficient plug-and-play framework, which directly combines an off-the-shelf LM (i.e., GPT-2) and an image-text matching model (i.e., CLIP) for image-grounded text generation. During decoding, MAGIC influences the generation of the LM by introducing a CLIP-induced score, called magic score, which regularizes the generated result to be semantically related to a given image while being coherent to the previously generated context. Notably, the proposed decoding scheme does not involve any gradient update operation, therefore being computationally efficient. On the challenging task of zero-shot image captioning, MAGIC outperforms the state-of-the-art method by notable margins with a nearly 27 times decoding speedup. MAGIC is a flexible framework and is theoretically compatible with any text generation tasks that incorporate image grounding. In the experiments, we showcase that it is also capable of performing visually grounded story generation given both an image and a text prompt. Our code, models and other related resources are publicly released at https://github.com/yxuansu/MAGIC.

show abstract

Section: A Related Workmentioning

confidence: 99%

Language Models Can See: Plugging Visual Controls in Text Generation

Su¹,

Lan²,

Liu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Data-to-Text Generation Recently, retrievalaugmented generation has been adapted to the task of data-to-text generation. To bridge the gap between the structured data and natural language text, Su et al (2021a) propose a novel retrievalaugmented framework. Specifically, given the source data, a set of candidate texts are first retrieved from a large unlabelled corpus.…”

Section: Text Style Transfermentioning

confidence: 99%

A Survey on Retrieval-Augmented Text Generation

Li¹,

Su²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Recently, retrieval-augmented text generation attracted increasing attention of the computational linguistics community. Compared with conventional generation models, retrievalaugmented text generation has remarkable advantages and particularly has achieved state-ofthe-art performance in many NLP tasks. This paper aims to conduct a survey about retrievalaugmented text generation. It firstly highlights the generic paradigm of retrieval-augmented generation, and then it reviews notable approaches according to different tasks including dialogue response generation, machine translation, and other generation tasks. Finally, it points out some promising directions on top of recent methods to facilitate future research.

show abstract

“…al. [172] adopt a BM25 [268] based information retrieval (IR) system to their prototype-to-generate (P2G) framework, which, aided with their BERT-based prototype selector, retrieves contextual samples for the input data instance from Wikipedia, allowing for successful few shot learning in T5. For reasoning over tabulated sport summaries, Li et.…”

Section: Regularization Techniquesmentioning

confidence: 99%

Innovations in Neural Data-to-text Generation

Sharma¹,

Gogineni²,

Ramakrishnan³

2022

Preprint

View full text Add to dashboard Cite

The neural boom that has sparked natural language processing (NLP) research through the last decade has similarly led to significant innovations in data-to-text generation (DTG). This survey offers a consolidated view into the neural DTG paradigm with a structured examination of the approaches, benchmark datasets, and evaluation protocols. This survey draws boundaries separating DTG from the rest of the natural language generation (NLG) landscape, encompassing an up-to-date synthesis of the literature, and highlighting the stages of technological adoption from within and outside the greater NLG umbrella. With this holistic view, we highlight promising avenues for DTG research that not only focus on the design of linguistically capable systems but also systems that exhibit fairness and accountability.Index Terms-Data-to-text generation (DTG), natural language generation (NLG) ! 1. This survey exclusively focuses on academic innovations for datato-text generation as the technologies underlying commercial frameworks are often proprietary.

show abstract

Few-Shot Table-to-Text Generation with Prototype Memory

Cited by 12 publications

References 22 publications

Language Models Can See: Plugging Visual Controls in Text Generation

Language Models Can See: Plugging Visual Controls in Text Generation

A Survey on Retrieval-Augmented Text Generation

Innovations in Neural Data-to-text Generation

Contact Info

Product

Resources

About