A Novel Word Segmentation Method Based on Object Detection and Deep Learning

Wilkinson, Tomas; Brun, Anders

doi:10.1007/978-3-319-27857-5_21

Cited by 8 publications

(8 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is a powerful and relatively light-weight model that achieves state-of-the-art results in segmentation-free word spotting. The Ctrl-F-Mini allows for searches using arbitrary text queries in full-page text images, and consists of a pre-activation ResNet34 backbone [16]; a bilinear interpolation layer that resizes region proposals into a canonical output size [17], [18]; an embedding network that embeds region proposals into a word embedding space; and external region proposals generated by the Dilated Text Proposals algorithm [19]. The output consists of a set of word proposals and corresponding scores that encode the probability of a proposal being centred on a word.…”

Section: A Word Spotting Modelmentioning

confidence: 99%

Bootstrapping Weakly Supervised Segmentation-free Word Spotting through HMM-based Alignment

Wilkinson

Nettelblad

2020

2020 17th International Conference on Frontiers in Handwriting Recognition (ICFHR)

Self Cite

View full text Add to dashboard Cite

Recent work in word spotting in handwritten documents has yielded impressive results. This progress has largely been made by supervised learning systems, which are dependent on manually annotated data, making deployment to new collections a significant effort. In this paper, we propose an approach that utilises transcripts without bounding box annotations to train segmentation-free query-by-string word spotting models, given a partially trained model. This is done through a training-free alignment procedure based on hidden Markov models. This procedure creates a tentative mapping between word region proposals and the transcriptions to automatically create additional weakly annotated training data, without choosing any single alignment possibility as the correct one. When only using between 1% and 7% of the fully annotated training sets for partial convergence, we automatically annotate the remaining training data and successfully train using it. On all our datasets, our final trained model then comes within a few mAP% of the performance from a model trained with the full training set used as ground truth. We believe that this will be a significant advance towards a more general use of word spotting, since digital transcription data will already exist for parts of many collections of interest.

show abstract

Section: A Word Spotting Modelmentioning

confidence: 99%

Bootstrapping Weakly Supervised Segmentation-free Word Spotting through HMM-based Alignment

Wilkinson

Nettelblad

2020

2020 17th International Conference on Frontiers in Handwriting Recognition (ICFHR)

Self Cite

View full text Add to dashboard Cite

show abstract

“…The contributions of this paper include: 1) Two models for segmentation-free query-by-string word spotting are introduced: An end-to-end trainable model based on Faster R-CNN [18] and previous work [19], [20]; and a simplified version that performs equally well or better in certain situations. 2) Two novel data augmentation strategies for full manuscript pages, crucial for preventing model overfitting.…”

Section: Contributionsmentioning

confidence: 99%

Neural Word Search in Historical Manuscript Collections

Wilkinson¹,

Lindstrøm²,

Brun³

2018

Preprint

Self Cite

View full text Add to dashboard Cite

We address the problem of segmenting and retrieving word images in collections of historical manuscripts given a text query. This is commonly referred to as "word spotting". To this end, we first propose an end-to-end trainable model based on deep neural networks that we dub Ctrl-F-Net. The model simultaneously generates region proposals and embeds them into a word embedding space, wherein a search is performed. We further introduce a simplified version called Ctrl-F-Mini. It is faster with similar performance, though it is limited to more easily segmented manuscripts. We evaluate both models on common benchmark datasets and surpass the previous state of the art. Finally, in collaboration with historians, we employ the Ctrl-F-Net to search within a large manuscript collection of over 100 thousand pages, written across two centuries. With only 11 training pages, we enable large scale data collection in manuscript-based historical research. This results in a speed up of data collection and the number of manuscripts processed by orders of magnitude. Given the time consuming manual work required to study old manuscripts in the humanities, quick and robust tools for word spotting has the potential to revolutionise domains like history, religion and language.

show abstract

“…As the RPN is a sliding window approach, complementing its proposals with an external region proposal method based on connected components is likely to improve the recall rate. To extract external proposals, we use the recently introduced method from [37], which is specially designed for manuscript images, and can be considered to be based on connected components. While the method is not given a name, we call it Dilated Text Proposals (DTP) to increase the clarity of this work.…”

Section: Dilated Text Proposalsmentioning

confidence: 99%

Neural Ctrl-F: Segmentation-Free Query-by-String Word Spotting in Handwritten Manuscript Collections

Wilkinson

Lindström

Brun

2017

2017 IEEE International Conference on Computer Vision (ICCV)

Self Cite

View full text Add to dashboard Cite

In this paper, we approach the problem of segmentationfree query-by-string word spotting for handwritten documents. In other words, we use methods inspired from computer vision and machine learning to search for words in large collections of digitized manuscripts. In particular, we are interested in historical handwritten texts, which are often far more challenging than modern printed documents. This task is important, as it provides people with a way to quickly find what they are looking for in large collections that are tedious and difficult to read manually. To this end, we introduce an end-to-end trainable model based on deep neural networks that we call Ctrl-F-Net. Given a full manuscript page, the model simultaneously generates region proposals, and embeds these into a distributed word embedding space, where searches are performed. We evaluate the model on common benchmarks for handwritten word spotting, outperforming the previous state-of-theart segmentation-free approaches by a large margin, and in some cases even segmentation-based approaches. One interesting real-life application of our approach is to help historians to find and count specific words in court records that are related to women's sustenance activities and division of labor. We provide promising preliminary experiments that validate our method on this task.

show abstract

A Novel Word Segmentation Method Based on Object Detection and Deep Learning

Cited by 8 publications

References 13 publications

Bootstrapping Weakly Supervised Segmentation-free Word Spotting through HMM-based Alignment

Bootstrapping Weakly Supervised Segmentation-free Word Spotting through HMM-based Alignment

Neural Word Search in Historical Manuscript Collections

Neural Ctrl-F: Segmentation-Free Query-by-String Word Spotting in Handwritten Manuscript Collections

Contact Info

Product

Resources

About