Metaknowledge Extraction Based on Multi-Modal Documents

Liu, Shu-Kan; Xu, Ruilin; Sun, Qiao; Duan, Li; Liu, Yiming

doi:10.1109/access.2021.3068728

Cited by 4 publications

(5 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Visual features can identify some regions (e.g., figures, tables), whereas textual features are critical to discriminate visually similar regions (e.g., keywords, abstract, affiliation, author names, etc.). However, single modality models have insufficient capability for layout modeling, hence multi-modal approaches have recently become more popular [9,10,37]. However, they typically contain only hundreds of labeled pages due to prohibitive labeling costs to annotate many layout objects per page, which is insufficient to train and evaluate deep learning based models [27].…”

Section: Vision-based Document Analysismentioning

confidence: 99%

“…Zheng et al [45] introduced four granularity levels for document modeling: documents, paragraphs, sentences, and tokens, reflecting the natural hierarchical document structure. More recently, reference [9] defined a document structure tree model to organize knowledge element extraction from documents and determine their relationships, such as juxtaposition and inclusive, between sections at different levels.…”

Section: Document Modelingmentioning

confidence: 99%

“…Then, researchers (or policymakers) can expedite their decision-making by streamlining information/knowledge collection and analysis. For example, reference [48] performed a behavioral study on citations, reference [3] extracted tasks, datasets, metrics, and scores from NLP papers to automatically construct a leaderboard, and reference [9] suggested a metaknowledge construction framework and document structure tree model to reduce gaps between human knowledge perception and entity-relationship triplets.…”

Section: Organizing Knowledge With Sekg For Multiple Documentsmentioning

confidence: 99%

“…However, their relationship extraction modules still only achieved an F1-score < 80%. Liu et al [9] defined a metaknowledge architecture to construct structural knowledge with documents, in contrast with previous KGs but similar to the present paper's approach. They employed a multi-modal metaknowledge extraction model to extract and organize metaknowledge elements (e.g., titles, authors, abstracts, and sections) from a government policy document dataset and DocBank [10] dataset.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Layout Aware Semantic Element Extraction for Sustainable Science & Technology Decision Support

et al. 2022

View full text Add to dashboard Cite

New scientific and technological (S&T) knowledge is being introduced rapidly, and hence, analysis efforts to understand and analyze new published S&T documents are increasing daily. Automated text mining and vision recognition techniques alleviate the burden somewhat, but the various document layout formats and knowledge content granularities across the S&T field make it challenging. Therefore, this paper proposes LA-SEE (LAME and Vi-SEE), a knowledge graph construction framework that simultaneously extracts meta-information and useful image objects from S&T documents in various layout formats. We adopt Layout-aware Metadata Extraction (LAME), which can accurately extract metadata from various layout formats, and implement a transformer-based instance segmentation (i.e., Vision based Semantic Elements Extraction (Vi-SEE)) to maximize the vision-based semantic element recognition. Moreover, to constructing a scientific knowledge graph consisting of multiple S&T documents, we newly defined an extensible Semantic Elements Knowledge Graph (SEKG) structure. For now, we succeeded in extracting about 6 million semantic elements from 49,649 PDFs. In addition, to illustrate the potential power of our SEKG, we provide two promising application scenarios, such as a scientific knowledge guide across multiple S&T documents and questions and answering over scientific tables.

show abstract

Section: Vision-based Document Analysismentioning

confidence: 99%

Section: Document Modelingmentioning

confidence: 99%

Section: Organizing Knowledge With Sekg For Multiple Documentsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Layout Aware Semantic Element Extraction for Sustainable Science & Technology Decision Support

et al. 2022

View full text Add to dashboard Cite

show abstract

“…• • • , D 5 }.For127 each document D i , we use open source NLP models to extract the entities and relations 128 (referred to as metaknowledge semantic elements in this work) in paragraphs.129In this work, we transform the HTML script of each Wiki document web page 130 into hierarchical XML files by parsing the HTML labels, such as <h1>, <h2>, <h3>, 131 <div id="toc"...>, <p>, which represent the title, section titles, summary or para-132 graphs (referred to as metaknowledge hierarchical elements in Ref [7]…”

mentioning

confidence: 99%

Metaknowledge Enhanced Open Domain Question Answering with Wiki Documents

Liu¹,

Xu²,

Duan³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The commonly-used large-scale knowledge bases have been facing challenges in open domain question answering tasks which are caused by the loose knowledge association and weak structural logic of triplet-based knowledge. To find a way out of this dilemma, this work proposes a novel metaknowledge enhanced approach for open domain question answering. We design an automatic approach to extract metaknowledge and build metaknowledge network from Wiki documents. For the purpose of representing the directional weighted graph with hierarchical and semantic features, we present an original graph encoder GE4MK to model the metaknowledge network. Then a metaknowledge enhanced graph reasoning model MEGr-Net is proposed for question answering, which aggregates both relational and neighboring interactions comparing with R-GCN and GAT. Experiments have proved the improvement of metaknowledge over main-stream triplet-based knowledge. We have found that the graph reasoning models and pre-trained language models also have influences on the metaknowledge enhanced question answering approaches.

show abstract

Table Detection & Data Extraction from Documents using Object Detection

Khan,

Durga

et al. 2023

2023 8th International Conference on Communication and Electronics Systems (ICCES)

View full text Add to dashboard Cite

Metaknowledge Extraction Based on Multi-Modal Documents

Cited by 4 publications

References 27 publications

Layout Aware Semantic Element Extraction for Sustainable Science & Technology Decision Support

Layout Aware Semantic Element Extraction for Sustainable Science & Technology Decision Support

Metaknowledge Enhanced Open Domain Question Answering with Wiki Documents

Table Detection & Data Extraction from Documents using Object Detection

Contact Info

Product

Resources

About