KnowSemLM: A Knowledge Infused Semantic Language Model

Peng, Haoruo; Ning, Qiang; Roth, Dan

doi:10.18653/v1/k19-1051

Cited by 13 publications

(5 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…4 Related Work 4.1 Schema Induction Chambers andJurafsky (2008, 2009) automatically learned a schema from newswire text based on coreference and statistical probability models. Peng and Roth (2016); Peng et al (2019) generated an event schema based on their proposed semantic language model., representing the whole schema as a linear sequence of abstract VerbNet (Schuler, 2005) verb senses. In these works, the schema was created for a single actor (protagonist).…”

Section: Metricmentioning

confidence: 99%

SAGEViz: SchemA GEneration and Visualization

Devare,

Koupaee,

Gunapati

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations

View full text Add to dashboard Cite

Schema induction involves creating a graph representation depicting how events unfold in a scenario. We present SAGEViz, an intuitive and modular tool that utilizes human-AI collaboration to create and update complex schema graphs efficiently, where multiple annotators (humans and models) can work simultaneously on a schema graph from any domain. The tool consists of two components: (1) a curation component powered by plug-and-play event language models to create and expand event sequences while human annotators validate and enrich the sequences to build complex hierarchical schemas, and (2) an easy-to-use visualization component to visualize schemas at varying levels of hierarchy. Using supervised and few-shot approaches, our event language models can continually predict relevant events starting from a seed event. We conduct a user study and show that users need less effort in terms of interaction steps with SAGEViz to generate schemas of better quality. We also include a video demonstrating the system 1 .

show abstract

Section: Metricmentioning

confidence: 99%

SAGEViz: SchemA GEneration and Visualization

Devare,

Koupaee,

Gunapati

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations

View full text Add to dashboard Cite

show abstract

“…Knowledge infusion is a common strategy [14,15] to handle low-resource learning tasks with limited supervision. In RE, knowledge infusion has also been found effective.…”

Section: Knowledge Infusion In Rementioning

confidence: 99%

Full-Abstract Biomedical Relation Extraction with Keyword-Attentive Domain Knowledge Infusion

Xiang

Zhang

Du³

et al. 2021

Applied Sciences

View full text Add to dashboard Cite

Relation extraction (RE) is an essential task in natural language processing. Given a context, RE aims to classify an entity-mention pair into a set of pre-defined relations. In the biomedical field, building an efficient and accurate RE system is critical for the construction of a domain knowledge base to support upper-level applications. Recent advances have witnessed a focus shift from sentence to document-level RE problems, which are more challenging due to the need for inter- and intra-sentence semantic reasoning. This type of distant dependency is difficult to understand and capture for a learning algorithm. To address the challenge, prior efforts either attempted to improve the cross sentence text representation or infuse domain or local knowledge into the model. Both strategies demonstrated efficacy on various datasets. In this paper, a keyword-attentive knowledge infusion strategy is proposed and integrated into BioBERT. A domain keyword collection mechanism is developed to discover the most relation-suggestive word tokens for bio-entities in a given context. By manipulating the attention masks, the model can be guided to focus on the semantic interaction between bio-entities linked by the keywords. We validated the proposed method on the Biocreative V Chemical Disease Relation dataset with an F1 of 75.6%, outperforming the state-of-the-art by 5.6%.

show abstract

“…Narrative Event Schema Induction. Previous work (Chambers and Jurafsky, 2008Jans et al, 2012;Balasubramanian et al, 2013;Mooney, 2014, 2016;Rudinger et al, 2015;Granroth-Wilding and Clark, 2016;Modi, 2016;Mostafazadeh et al, 2016a;Peng et al, 2019) focuses on inducing narrative schemas as partially ordered sets of events (represented as verbs) sharing a common argument. The event order is further extended to include causality (Mostafazadeh et al, 2016b;Kalm et al, 2019), and temporal script graph is proposed where events and arguments are abstracted as event types and participant types (Modi et al, 2017;Wanzare et al, 2017;Zhai et al, 2019).…”

Section: Related Workmentioning

confidence: 99%

Connecting the Dots: Event Graph Schema Induction with Path Language Modeling

Zeng

Lin

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

Event schemas can guide our understanding and ability to make predictions with respect to what might happen next. We propose a new Event Graph Schema, where two event types are connected through multiple paths involving entities that fill important roles in a coherent story. We then introduce Path Language Model, an auto-regressive language model trained on event-event paths, and select salient and coherent paths to probabilistically construct these graph schemas. We design two evaluation metrics, instance coverage and instance coherence, to evaluate the quality of graph schema induction, by checking when coherent event instances are covered by the schema graph. Intrinsic evaluations show that our approach is highly effective at inducing salient and coherent schemas. Extrinsic evaluations show the induced schema repository provides significant improvement to downstream end-to-end Information Extraction over a state-of-the-art joint neural extraction model, when used as additional global features to unfold instance graphs. 1 Event Instance GraphsPath Language Model PER Transport Attack artifact attacker Transport GPE PER agent affliation Attack attacker Transport GPE origin WEA artifact instrument Attack Transport WEA artifact instrument Attack Transport FAC Attack GPE place partwhole destination . . .

show abstract

KnowSemLM: A Knowledge Infused Semantic Language Model

Cited by 13 publications

References 52 publications

SAGEViz: SchemA GEneration and Visualization

SAGEViz: SchemA GEneration and Visualization

Full-Abstract Biomedical Relation Extraction with Keyword-Attentive Domain Knowledge Infusion

Connecting the Dots: Event Graph Schema Induction with Path Language Modeling

Contact Info

Product

Resources

About