Enhancing document structure analysis using visual analytics

Abstract

During the last decade national archives, libraries, museums and companies started to make their records, books and files electronically available. In order to allow efficient access of this information, the content of the documents must be stored in database and information retrieval systems. State-of-the-art indexing techniques mostly rely on the information explicitly available in the text portions of documents. Documents usually contain a significant amount of implicit information such as their logical structure which is not directly accessible (unless the documents are available as well-structured XML-files) and is therefore not used in the search process. In this paper, a new approach for analyzing the logical structure of text documents is presented. The problem of state-of-the-art methods is that they have been developed for a particular type of documents and can only handle documents of that type. In most cases, adaptation and re-training for a different document type is not possible. Our proposed method allows an efficient and effective adaptation of the structure analysis process by combining state-of-the-art machine learning with novel interactive visualization techniques, allowing a quick adaptation of the structure analysis process to unknown document classes and new tasks without requiring a predefined training set.

ICDAR 2013 Table Competition

“…Andreas Stoffel, from the Department of Computer and Information Science, University of Konstanz, Germany, participated with a trainable system [9], [10] for the analysis of PDF documents based on the PDFBox library. After initial column and reading-order detection, logical classification is performed on the line level.…”

A Novel Approach for Document Retrieval System with User Preferences

“…The Information retrieval system was integrated with a text editor in order to find similar documents. They analyzed the extraction of logical structure from different text document formats [4], [5] and also from source code documents. The second area was the extraction of semantically coherent blocks of text from documents [6].…”

Using structural information and citation evidence to detect significant plagiarism cases in scientific publications

“…Partitioning scholarly documents comes under a wide research problem know as logical structure extraction (LSE) of semistructured documents, and is not the focus of this work. Fortunately, there are efficient LSE solutions addressed in recent literature (Burget, 2007; Luong et al, 2010; Ratté et al, 2007; Stoffel et al, 2010). In this work, we employ Luong's LSE (Luong et al, 2010) developed by the National University of Singapore (NUS), and available for free use or adaptation within other tools under the Lesser GNU Public License (LGPL) 5…”

“…Structural components are subject to interpretation by the reader, but also can be identified automatically using LSE methods (Burget, 2007; Luong et al, 2010; Ratté et al, 2007; Stoffel et al, 2010). In this work, we use SectLabel tool (Luong et al, 2010) to extract the logical structure of scientific publications.…”

“…These segments of scholarly documents, commonly referred to in the literature as logical structure, can be extracted (Anjewierden, 2001; Bounhas & Slimani, 2010; Burget, 2007; Councill, Giles, & Kan, 2008; Hagen, Harald, Ngen, & Petra Saskia, 2004; K.H. Lee, Choy, & Cho, 2003; Li & Ng, 2004; Luong, Nguyen, & Kan, 2010; Nguyen & Luong, 2010; Ratté, Njomgue, & Ménard, 2007; Stoffel, Spretke, Kinnemann, & Keim, 2010; Wang, Jin, Wang, Wang, & Gao, 2005; Witt et al, 2010; K. Zhang, Wu, & Li, 2006), and can be used to improve document indexing (Bounhas & Slimani, 2010), to represent the semantic content of scientific publications (Luong, Nguyen, & Kan, 2010; Ratté et al, 2007), to extract key phrases and terminologies (Bounhas & Slimani, 2010; Nguyen & Luong, 2010), and to improve document summarization (Teufel & Moens, 2002). To improve indexing of semistructured documents, for instance, a method of terms weighting is applied according to their structural occurrences (or their positions in different segments of the document), instead of using the whole document as in flat weighting methods (Bounhas & Slimani, 2010; de Moura, Fernandes, Ribeiro‐Neto, da Silva, & Gonçalves, 2010).…”