Reasoning about Procedures with Natural Language Processing: A Tutorial

Zhang, Li

doi:10.48550/arxiv.2205.07455

Cited by 2 publications

(2 citation statements)

References 15 publications

(27 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Procedural Texts Procedural texts are an attractive data source to reason about events and entities which undergo frequent changes. There has been steady efforts in computer vision (Miech et al, 2019), robotics (Ahn et al, 2022), and language (Mujtaba and Mahapatra, 2019;Zhang, 2022). In NLP specifically, work on procedures includes extracting them from instructional texts (Paris et al, 2002;Delpech and Saint-Dizier, 2008;Zhang et al, 2012), reasoning about events (Takechi et al, 2003;Rajagopal et al, 2020;Zhang et al, 2020c), knowledge-base construction (Jung et al, 2010;Chu et al, 2017;Park and Motahari Nezhad, 2018), or applying them to downstream applications (Yang et al, 2021b,a;Zhang et al, 2020a;Lyu et al, 2021;Zhang et al, 2020b;Chen et al, 2020).…”

Section: Code-based Language Models and Promptsmentioning

confidence: 99%

“…In the previous cooking example, whether "there is a sizzling sound" depends on what steps have taken place. With these interesting challenges coupled with the added benefit of application to robotics and household smart assistants such as Alexa (Panagopoulou et al, 2022), reasoning about procedures attracts great attention from the NLP community (Zhang, 2022).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Causal Reasoning of Entities and Events in Procedural Texts

Zhang,

Xu,

Yang

et al. 2023

Findings of the Association for Computational Linguistics: EACL 2023

View full text Add to dashboard Cite

Entities and events are crucial to natural language reasoning and common in procedural texts. Existing work has focused either exclusively on entity state tracking (e.g., whether a pan is hot) or on event reasoning (e.g., whether one would burn themselves by touching the pan), while these two tasks are often causally related. We propose CREPE, the first benchmark on causal reasoning of event plausibility and entity states. We show that most language models, including GPT-3, perform close to chance at .35 F1, lagging far behind human at .87 F1. We boost model performance to .59 F1 by creatively representing events as programming languages while prompting language models pretrained on code. By injecting the causal relations between entities and events as intermediate reasoning steps in our representation, we further boost the performance to .67 F1. Our findings indicate not only the challenge that CREPE brings for language models, but also the efficacy of code-like prompting combined with chain-of-thought prompting for multihop event reasoning. 1 * Equal contribution. 1 Data and code can be found at https://github.com/z harry29/causal_reasoning_of_entities_and_events.

show abstract