TeLCoS: OnDevice Text Localization with Clustering of Script

Munjal, Rachit S; Goyal, Manoj; Moharir, Rutika; Moharana, Sukumar

doi:10.1109/ijcnn52387.2021.9533292

Cited by 5 publications

(2 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Rectifying and pre-processing each word crop adds to the computation cost of the network. To handle this trade-off between accuracy and inference time, we extract rotation angle and skewness information from the localization module [26] and pass only the highly rotated words through a computer vision-based perspective transformation. The model is made invariant to small amounts of distortions in the text.…”

Section: A Selective Rotationmentioning

confidence: 99%

STRIDE : Scene Text Recognition In-Device

Munjal,

Prabhu,

Arora

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Optical Character Recognition (OCR) systems have been widely used in various applications for extracting semantic information from images. To give the user more control over their privacy, an on-device solution is needed. The current state of the art models are too heavy and complex to be deployed on-device. We develop an efficient lightweight scene text recognition (STR) system, which has only 0.88M parameters and performs real-time text recognition. Attention modules tend to boost the accuracy of STR networks but are generally slow and not optimized for device inference. So, we propose the use of convolution attention modules to the text recognition networks, which aims to provide channel and spatial attention information to the LSTM module by adding very minimal computational cost. It boosts our word accuracy on ICDAR 13 dataset by almost 2%. We also introduce a novel orientation classifier module, to support the simultaneous recognition of both horizontal and vertical text.The proposed model surpasses on-device metrics of inference time and memory footprint and achieves comparable accuracy when compared to the leading commercial and other open-source OCR engines. We deploy the system on-device with an inference speed of 2.44 ms per word on the Exynos 990 chipset device and achieve an accuracy of 88.4% on ICDAR-13 dataset.

show abstract

Section: A Selective Rotationmentioning

confidence: 99%

STRIDE : Scene Text Recognition In-Device

Munjal,

Prabhu,

Arora

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…We create a very light-weight CNN model [19] with a Unet architecture to perform Scene text detection on the mobile screenshot and natural images. It supports the detection of multi-oriented, multi-scale scene text as well as incidental text.…”

Section: B Text Block Identificationmentioning

confidence: 99%

ScreenSeg: On-Device Screenshot Layout Analysis

Goyal¹,

Munjal²,

Moharana³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We propose a novel end-to-end solution that performs a Hierarchical Layout Analysis of screenshots and document images on resource constrained devices like mobilephones. Our approach segments entities like Grid, Image, Text and Icon blocks occurring in a screenshot. We provide an option for smart editing by auto highlighting these entities for saving or sharing. Further, this multi-level layout analysis of screenshots has many use cases including content extraction, keyword-based image search, style transfer, etc. We have addressed the limitations of known baseline approaches, supported a wide variety of semantically complex screenshots, and developed an approach that is highly optimized for on-device deployment. In addition, we present a novel weighted NMS technique for filtering object proposals. We achieve an average precision of about 0.95 with a latency of around 200ms on the Samsung Galaxy S10 Device for a screenshot of 1080p resolution. The solution pipeline is already commercialized in Samsung Device applications i.e. Samsung Capture, Smart Crop, My Filter in Camera Application, Bixby Touch.

show abstract