Document processing for automatic knowledge acquisition

Tang, Yuan Yan; Yan, Chang; Suen, Ching Y.

doi:10.1109/69.273022

Cited by 77 publications

(2 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While computer vision models for extracting text from images have existed for decades (LeCun et al [1989], Tang et al [1994Tang et al [ , 1996, Sinha and Bansal [1995]), the development of new, robust neural networks for text layout, detection and recognition is still an active area of research (Xu et al [2020], Li et al [2021]), particularly in the context of text in natural scenes (Bartz et al [2018]).…”

Section: Introductionmentioning

confidence: 99%

Sim2Real Docs: Domain Randomization for Documents in Natural Scenes using Ray-traced Rendering

Maddikunta¹,

Zhao²,

Keswani³

et al. 2021

Preprint

View full text Add to dashboard Cite

In the past, computer vision systems for digitized documents could rely on systematically captured, high-quality scans. Today, transactions involving digital documents are more likely to start as mobile phone photo uploads taken by non-professionals. As such, computer vision for document automation must now account for documents captured in natural scene contexts. An additional challenge is that task objectives for document processing can be highly use-case specific, which makes publiclyavailable datasets limited in their utility, while manual data labeling is also costly and poorly translates between use cases.To address these issues we created Sim2Real Docs 2 -a framework for synthesizing datasets and performing domain randomization of documents in natural scenes. Sim2Real Docs enables programmatic 3D rendering of documents using Blender, an open source tool for 3D modeling and ray-traced rendering. By using rendering that simulates physical interactions of light, geometry, camera, and background, we synthesize datasets of documents in a natural scene context. Each render is paired with use-case specific ground truth data specifying latent characteristics of interest, producing unlimited fit-for-task training data. The role of machine learning models is then to solve the inverse problem posed by the rendering pipeline. Such models can be further iterated upon with real-world data by either fine tuning or making adjustments to domain randomization parameters. Keywords computer vision• synthetic data • data centric ai • document processing • text detection • text recognition • data augmentation • natural scenes

show abstract

Section: Introductionmentioning

confidence: 99%

Sim2Real Docs: Domain Randomization for Documents in Natural Scenes using Ray-traced Rendering

Maddikunta¹,

Zhao²,

Keswani³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…During the '80s, knowledge acquisition research was boosted thanks to the development of expert systems [104]. In the '90s, Tang et al [258] had already compiled eight computer-based techniques for acquiring data and knowledge. Text documents are typically used as the source of automated knowledge extraction [98,124,176,228].…”

Section: The Problem and Motivationmentioning

confidence: 99%

Extraction of Knowledge Models from Textbooks

Chacón¹

View full text Add to dashboard Cite

Many adaptive educational systems and other artificial intelligence applications rely on high-quality knowledge representations. Still, knowledge acquisition remains the primary bottleneck hindering large-scale deployment and adoption of knowledge-based systems. One path to scalable knowledge extraction is using digital textbooks, given their domain-oriented content, structure, and availability. This dissertation presents a unified approach for automatically extracting high-quality and domain-specific knowledge models from digital textbooks. The proposed approach leverages the authors’ knowledge encoded in the textbooks’ elements that facilitate navigation and understanding of the material (table of contents, index, formatting styles) to create knowledge models. The proposed workflow first extracts initial information elements from the textbooks: the structure of chapters and subchapters using the Table of Content, the content of each section, and domain terminology from the back-of-the-book index. Then, new information is added: domain terms are linked to external entities in a knowledge graph (DBpedia) and are enriched with semantic content (e.g., abstracts and categories). Finally, the knowledge about the domain is refined by identifying the relevance of concepts to the target domain. The extracted knowledge is represented in a model using the Text Encoding Initiative. Multiple evaluations show that the extracted knowledge models have high levels of quality across several properties: accuracy, semantics, coverage, specificity, cognitive validity, and granularity. Additionally, the approach is effective in multiple domains—for example, statistics, ancient philosophy, and Python programming. Finally, there are many potential applications for the extracted knowledge models. This dissertation presents three different educational systems supported by the knowledge models.

show abstract