Word Spotting and Recognition Using Deep Embedding

Krishnan, Praveen; Dutta, Kartik; Jawahar, C. V.

doi:10.1109/das.2018.70

Cited by 86 publications

(82 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this application, text and symbols are treated as noise that must be removed, but the use is planned of these features as information for the automatic creation of map labels as points in a vector map. The word spotting technique [74] will be tested with the query-by-string (QBS) approach [62,75], using the available list of toponyms as input. This approach has already been tested for maps, but text portions on the maps are obtained through image binarization [76,77], usually applying Otsu's global thresholding [78], rather than segmentation, and the focus is on words recognition rather than the separation of areas and labels on the map.…”

Section: Transects In Bluementioning

confidence: 99%

New Tools for the Classification and Filtering of Historical Maps

Gobbi

Ciolli

Porta

et al. 2019

IJGI

View full text Add to dashboard Cite

Historical maps constitute an essential information for investigating the ecological and landscape features of a region over time. The integration of heritage maps in GIS models requires their digitalization and classification. This paper presents a semi-automatic procedure for the digitalization of heritage maps and the successive filtering of undesirable features such as text, symbols and boundary lines. The digitalization step is carried out using Object-based Image Analysis (OBIA) in GRASS GIS and R, combining image segmentation and machine-learning classification. The filtering step is performed by two GRASS GIS modules developed during this study and made available as GRASS GIS add-ons. The first module evaluates the size of the filter window needed for the removal of text, symbols and lines; the second module replaces the values of pixels of the category to be removed with values of the surrounding pixels. The procedure has been tested on three maps with different characteristics, the “Historical Cadaster Map for the Province of Trento” (1859), the “Italian Kingdom Forest Map” (1926) and the “Map of the potential limit of the forest in Trentino” (1992), with an average classification accuracy of 97%. These results improve the performance of classification of heritage maps compared to more classical methods, making the proposed procedure that can be applied to heterogeneous sets of maps, a viable approach.

show abstract

Section: Transects In Bluementioning

confidence: 99%

New Tools for the Classification and Filtering of Historical Maps

Gobbi

Ciolli

Porta

et al. 2019

IJGI

View full text Add to dashboard Cite

show abstract

“…In order to show the generic nature of the proposed architecture and its extension for query-by-string (QBS) spotting, in our set of parallel works [35,36], we have used HWNet architecture for both embedding into word attribute space defined by phoc [35] and also proposed an end2end architecture [36] which learns a common subspace between a text and image modality. It enables both QBE and QBS based word spotting along with word recognition using a fixed lexicon.…”

Section: Query-by-string Spotting Resultsmentioning

confidence: 99%

“…In the later set of works [75,76] from the same group, PHOCNet was adapted with temporal pooling layer (TPP-PHOCNet) and evaluated under different loss functions and optimization algorithms which further improved the word spotting performance. In [35,36], the features computed from HWNet [37] are embedded into word attribute space by training attribute based svm classifiers and projecting both image and textual attributes to a common subspace. In [83], the authors propose a two stage architecture where a triplet cnn network is trained to reduce the distance between the anchor word image and a similar labeled (positive) word image, while simultaneously increasing the distance between the anchor and negative labeled word image.…”

Section: Deep Learningmentioning

confidence: 99%

“…In terms of architecture advancements, we use a much deeper network using residual connections and present variable sized images without distorting the aspect ratio of word images. The learned representation can be used for word attribute learning [35,36], spotting, and retrieval tasks for both handwritten and printed documents irrespective of the vocabulary used while training.…”

Section: Deep Learningmentioning

confidence: 99%

See 1 more Smart Citation

HWNet v2: an efficient word image representation for handwritten documents

Krishnan

Jawahar

2019

IJDAR

Self Cite

View full text Add to dashboard Cite

We present a framework for learning an efficient holistic representation for handwritten word images. The proposed method uses a deep convolutional neural network with traditional classification loss. The major strengths of our work lie in: (i) the efficient usage of synthetic data to pre-train a deep network, (ii) an adapted version of the ResNet-34 architecture with the region of interest pooling (referred to as HWNet v2) which learns discriminative features for variable sized word images, and (iii) a realistic augmentation of training data with multiple scales and distortions which mimics the natural process of handwriting. We further investigate the process of transfer learning to reduce the domain gap between synthetic and real domain, and also analyze the invariances learned at different layers of the network using visualization techniques proposed in the literature.Our representation leads to a state-of-the-art word spotting performance on standard handwritten datasets and historical manuscripts in different languages with minimal representation size. On the challenging iam dataset, our method is first to report an mAP of around 0.90 for word spotting with a representation size of just 32 dimensions. Furthermore, we also present results on printed document datasets in English and Indic scripts which validates the generic nature of the proposed framework for learning word image representation.

show abstract

“…This task of searching for words in images either by string or by example is called word spotting [7]. In recent years, the performance of word spotting systems has greatly improved through the use of learning based approaches using convolutional neural networks (CNNs), for example in the work by Sudholt and Fink [25] or by Krishnan et al [14]. However, one drawback of these approaches is the large amount of labeled data required to train these approaches.…”

Section: Introductionmentioning

confidence: 99%

Representative Image Selection for Data Efficient Word Spotting

Westphal

Grahn

Lavesson

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

This paper compares three different word image representations as base for label free sample selection for word spotting in historical handwritten documents. These representations are a temporal pyramid representation based on pixel counts, a graph based representation, and a pyramidal histogram of characters (PHOC) representation predicted by a PHOCNet trained on synthetic data. We show that the PHOC representation can help to reduce the amount of required training samples by up to 69% depending on the dataset, if it is learned iteratively in an active learning like fashion. While this works for larger datasets containing about 1 700 images, for smaller datasets with 100 images, we find that the temporal pyramid and the graph representation perform better.

show abstract

Word Spotting and Recognition Using Deep Embedding

Cited by 86 publications

References 12 publications

New Tools for the Classification and Filtering of Historical Maps

New Tools for the Classification and Filtering of Historical Maps

HWNet v2: an efficient word image representation for handwritten documents

Representative Image Selection for Data Efficient Word Spotting

Contact Info

Product

Resources

About