Digitizing a Million Books: Challenges for Document Analysis

Sankar, K. Pramod; Ambati, Vamshi; Pratha, Lakshmi; Jawahar, C. V.

doi:10.1007/11669487_38

Cited by 30 publications

(18 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The consequent shift from paper to digital support for documents provided new opportunities for their management and processing activities, solving the problems of duplication and sharing that seriously affected legacy (paper) documents. However, making document production easy and cheap, it also introduced new problems, consisting in a huge amount of available documents (and in an associated decrease of their content quality) (Sankar, 2006). The wellknown information overload problem, indeed, consists of the users' difficulty in accessing interesting information in large and heterogeneous repositories.…”

Section: The Document Processing Domainmentioning

confidence: 99%

FOL Learning for Knowledge Discovery in Documents

et al. 2012

View full text Add to dashboard Cite

This chapter proposes the application of machine learning techniques, based on first-order logic as a representation language, to the real-world application domain of document processing. First, the tasks and problems involved in document processing are presented, along with the prototypical system DOMINUS and its architecture, whose components are aimed at facing these issues. Then, a closer look is provided for the learning component of the system, and the two sub-systems that are in charge of performing supervised and unsupervised learning as a support to the system performance. Finally, some experiments are reported that assess the quality of the learning performance. This is intended to prove to researchers and practitioners of the field that first-order logic learning can be a viable solution to tackle the domain complexity, and to solve problems such as incremental evolution of the document repository.

show abstract

Section: The Document Processing Domainmentioning

confidence: 99%

FOL Learning for Knowledge Discovery in Documents

et al. 2012

View full text Add to dashboard Cite

show abstract

“…Scanned document images contain a large number of artifacts, which are cleaned on a large scale using a a semi-automatic process [3], by using various image processing operations. Owing to the variation in quality across the images, a single setup of image processing parameters would not be suitable for all.…”

Section: Issues In Scanningmentioning

confidence: 99%

“…These include the Universal Digital Library (UDL) [1], Digital Library of India (DLI) [2], Google Books, etc. [3]. Much effort is being put into the digitisation of massive quantities of documents.…”

Section: Introductionmentioning

confidence: 99%

Enabling Search over Large Collections of Telugu Document Images – An Automatic Annotation Based Approach

Sankar

Jawahar

2006

Computer Vision, Graphics and Image Processing

Self Cite

View full text Add to dashboard Cite

Abstract. For the first time, search is enabled over a massive collection of 21 Million word images from digitized document images. This work advances the state-of-the-art on multiple fronts: i) Indian language document images are made searchable by textual queries, ii) interactive content-level access is provided to document images for search and retrieval, iii) a novel recognition-free approach, that does not require an OCR, is adapted and validated iv) a suite of image processing and pattern classification algorithms are proposed to efficiently automate the process and v) the scalability of the solution is demonstrated over a large collection of 500 digitised books consisting of 75,000 pages.Character recognition based approaches yield poor results for developing search engines for Indian language document images, due to the complexity of the script and the poor quality of the documents. Recognition free approaches, based on word-spotting, are not directly scalable to large collections, due to the computational complexity of matching images in the feature space. For example, if it requires 1 mSec to match two images, the retrieval of documents to a single query, from a large collection like ours, would require close to a day's time. In this paper we propose a novel automatic annotation based approach to provide textual description of document images. With a one time, offline computational effort, we are able to build a text-based retrieval system, over annotated images. This system has an interactive response time of about 0.01 second. However, we pay the price in the form of massive offline computation, which is performed on a cluster of 35 computers, for about a month. Our procedure is highly automatic, requiring minimal human intervention.

show abstract

“…These include Google Books, the Universal Digital Library (UDL) [1] and the Digital Library of India (DLI) [2,25]. A large percentage of these collections are in non-Latin scripts.…”

Section: Introductionmentioning

confidence: 99%

Nearest neighbor based collection OCR

Sankar

Jawahar

Manmatha

2010

Proceedings of the 9th IAPR International Workshop on Document Analysis Systems

Self Cite

View full text Add to dashboard Cite

Conventional optical character recognition (OCR) systems operate on individual characters and words, and do not normally exploit document or collection context. We describe a Collection OCR which takes advantage of the fact that multiple examples of the same word (often in the same font) may occur in a document or collection. The idea here is that an OCR or a reCAPTCHA like process generates a partial set of recognized words. In the second stage, a nearest neighbor algorithm compares the remaining word-images to those already recognized and propagates labels from the nearest neighbors. It is shown that by using an approximate fast nearest neighbor algorithm based on Hierarchical K-Means (HKM), we can do this accurately and efficiently. It is also shown that profile based features perform much better than SIFT and Pyramid Histogram of Gradient (PHOG) features. We believe that this is because profile features are more robust to word degradations (common in our documents). This approach is applied to a collection of Telugu booksa language for which no commercial OCR exists. We show from a selection of 33 Telugu books that starting with OCR labels for only 30% of the collection we can recognize the remaining 70% of the words in the collection with 70% accuracy using this approach. Since the approach makes no language specific assumptions, it should be applicable to a large number of languages. In particular we are interested in its applicability to Indic languages and scripts.

show abstract

Digitizing a Million Books: Challenges for Document Analysis

Cited by 30 publications

References 6 publications

FOL Learning for Knowledge Discovery in Documents

FOL Learning for Knowledge Discovery in Documents

Enabling Search over Large Collections of Telugu Document Images – An Automatic Annotation Based Approach

Nearest neighbor based collection OCR

Contact Info

Product

Resources

About