A framework to access handwritten information within large digitized paper collections

Abstract

Abstract-We describe our efforts with the National Archives and Records Administration (NARA) to provide a form of automated search of handwritten content within large digitized document archives. With a growing push towards the digitization of paper archives there is an imminent need to develop tools capable of searching the resulting unstructured image data as data from such collections offer valuable historical records that can be mined for information pertinent a number of fields from the geosciences to the humanities. To carry out the search we use a Computer Vision technique called Word Spotting. A form of content based image retrieval, it avoids the still difficult task of directly recognizing the text by allowing a user to search using a query image containing handwritten text and ranking a database of images in terms of those that contain more similar looking content. In order to make this search capability available on an archive three computationally expensive pre-processing steps are required. We describe these steps, the open source framework we have developed, and how it can be used not only on the recently released 1940 Census data containing nearly 4 million high resolution scanned forms, but also on other collections of forms. With a growing demand to digitize our wealth of paper archives we see this type of automated search as a low cost scalable alternative to the costly manual transcription that would otherwise be required.

Digitization and search: A non-traditional use of HPC

“…For detailed results about the quality of the Word Spotting approach with and without the use of cluster trees, see [12].…”

“…Finally, the proposed framework is presented and experimental results are discussed. A detailed account of the first two pre-processing steps and the entire framework can be found in [12].…”

Using Lucene to index and search the digitized 1940 US Census

“…Here, we used the same features as in , specifically the upper, lower, and total column ink profiles. A detailed description of the feature extraction process, including the segmentation of the forms, can be found in .…”

“…Word Spotting feature vectors were previously suggested as a good descriptor to the handwritten content of images . In a previous work , we described a framework for using Word Spotting to provide automatic searchable access to the 1940 Census. The framework provides a means of generating Word Spotting feature vectors to describe the contents of hand‐filled forms, indexing these vectors to enable later search, and leveraging the users' activity in the system as a passive crowdsourcing element to improve the search results (Figure ).…”

“…The previously reported index was composed of a collection of binary trees where the nodes of the trees represented clusters of elements. While it achieved good results for the provided annotated testbed and real time evaluations, the method used to build this index proved to be overly time consuming when dealing with large datasets . We discuss here how to index the system with Apache Lucene, a Java library that enables textual search, adapting the approaches provided in and .…”