GAIA: A Fine-grained Multimedia Knowledge Extraction System

Li, Manling; Zareian, Alireza; Lin, Ying; Pan, Xiaoman; Whitehead, Spencer; Chen, Brian; Wu, Bo; Ji, Heng; Chang, Shih-Fu; Voss, Clare R.; Napierski, Daniel; Freedman, Marjorie R.

doi:10.18653/v1/2020.acl-demos.11

Cited by 56 publications

(38 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Text Information Extraction. Existing end-toend Information Extraction (IE) systems (Wadden et al, 2019;Li et al, 2020b;Li et al, 2019) mainly focus on extracting entities, events and entity relations from individual sentences. In contrast, we extract and infer arguments over the global document context.…”

Section: Related Workmentioning

confidence: 99%

“…Multimedia Information Extraction. Previous multimedia IE systems (Li et al, 2020b;Yazici et al, 2018) only include cross-media entity coreference resolution by grounding the extracted visual entities to text. We are the first to perform crossmedia joint event extraction and coreference resolution to obtain the coreferential events from text, images and videos.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

RESIN: A Dockerized Schema-Guided Cross-document Cross-lingual Cross-media Information Extraction and Event Tracking System

Wen¹,

Lin²,

Lai³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

Self Cite

View full text Add to dashboard Cite

We present a new information extraction system that can automatically construct temporal event graphs from a collection of news documents from multiple sources, multiple languages (English and Spanish for our experiment), and multiple data modalities (speech, text, image and video). The system advances state-of-the-art from two aspects: (1) extending from sentence-level event extraction to cross-document cross-lingual cross-media event extraction, coreference resolution and temporal event tracking; (2) using human curated event schema library to match and enhance the extraction output. We have made the dockerlized system publicly available for research purpose at GitHub 1 , with a demo video 2 .

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

RESIN: A Dockerized Schema-Guided Cross-document Cross-lingual Cross-media Information Extraction and Event Tracking System

Wen¹,

Lin²,

Lai³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

Self Cite

View full text Add to dashboard Cite

show abstract

“…15 Each given document may be in English, Russian, or Spanish. On a development set consisting solely of text-only documents, 16 we started with initial predictions made by GAIA (Li et al, 2020), for entity clusters, entity types, events and relations. Our goal was to recluster and relabel the a dataset for knowledge extraction.…”

Section: And Match Modules: Sm-kbpmentioning

confidence: 99%

LOME: Large Ontology Multilingual Extraction

Xia

Qin

Vashishtha

et al. 2021

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrati

View full text Add to dashboard Cite

We present LOME, a system for performing multilingual information extraction. Given a text document as input, our core system identifies spans of textual entity and event mentions with a FrameNet (Baker et al., 1998) parser. It subsequently performs coreference resolution, fine-grained entity typing, and temporal relation prediction between events. By doing so, the system constructs an event and entity focused knowledge graph. We can further apply third-party modules for other types of annotation, like relation extraction. Our (multilingual) first-party modules either outperform or are competitive with the (monolingual) state-of-the-art. We achieve this through the use of multilingual encoders like XLM-R (Conneau et al., 2020) and leveraging multilingual training data. LOME is available as a Docker container on Docker Hub. In addition, a lightweight version of the system is accessible as a web demo. * Equal Contribution 1 Information on using the Docker container, web demo, and demo video at https://nlp.jhu.edu/demos.

show abstract

“…More recently, and treat document-level event extraction as a templatefilling task. Li et al (2020a) performs event mention extraction and the two coreference tasks independently using a pipeline approach. However, none of the previous works learn entity and event coreference jointly with event mention extraction.…”

Section: Event Coreference Entity Coreferencementioning

confidence: 99%

Proceedings of the Third Workshop on Narrative Understanding

2021

View full text Add to dashboard Cite

While natural language understanding of longform documents remains an open challenge, such documents often contain structural information that can inform the design of models encoding them. Movie scripts are an example of such richly structured text -scripts are segmented into scenes, which decompose into dialogue and descriptive components. In this work, we propose a neural architecture to encode this structure, which performs robustly on two multi-label tag classification tasks without using handcrafted features. We add a layer of insight by augmenting the encoder with an unsupervised 'interpretability' module, which can be used to extract and visualize narrative trajectories. Though this work specifically tackles screenplays, we discuss how the underlying approach can be generalized to a range of structured documents.

show abstract

GAIA: A Fine-grained Multimedia Knowledge Extraction System

Cited by 56 publications

References 29 publications

RESIN: A Dockerized Schema-Guided Cross-document Cross-lingual Cross-media Information Extraction and Event Tracking System

RESIN: A Dockerized Schema-Guided Cross-document Cross-lingual Cross-media Information Extraction and Event Tracking System

LOME: Large Ontology Multilingual Extraction

Proceedings of the Third Workshop on Narrative Understanding

Contact Info

Product

Resources

About