Two Stream Deep Network for Document Image Classification

Asim, Muhammad Nabeel; Khan, Muhammad Usman Ghani; Malik, Muhammad Imran; Khizar, Razzaque; Dengel, Andreas; Ahmed, Sheraz

doi:10.1109/icdar.2019.00227

Cited by 30 publications

(39 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In order to get the final enhanced prediction of the combination of both text and image model we use a simple ensemble as in [5]. In this work w 1 , w 2 = 0.5 are found optimal.…”

Section: Image and Text Ensemblementioning

confidence: 99%

“…Recently, textual information has been used by itself or as a combination together with visual features extracted by the previously mentioned models. Although Optical Character Recognition (OCR) is prone to errors, particularly when dealing with handwritten documents, the use of modern Natural Language Processing (NLP) techniques have demonstrated a boost in the classifiers performance [5,6,35].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Improving Accuracy and Speeding Up Document Image Classification Through Parallel Systems

Ferrando

Domínguez

Torres

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

This paper presents a study showing the benefits of the Effi-cientNet models compared with heavier Convolutional Neural Networks (CNNs) in the Document Classification task, essential problem in the digitalization process of institutions. We show in the RVL-CDIP dataset that we can improve previous results with a much lighter model and present its transfer learning capabilities on a smaller in-domain dataset such as Tobacco3482. Moreover, we present an ensemble pipeline which is able to boost solely image input by combining image model predictions with the ones generated by BERT model on extracted text by OCR. We also show that the batch size can be effectively increased without hindering its accuracy so that the training process can be sped up by parallelizing throughout multiple GPUs, decreasing the computational time needed. Lastly, we expose the training performance differences between PyTorch and Tensorflow Deep Learning frameworks.

show abstract

“…In order to get the final enhanced prediction of the combination of both text and image model we use a simple ensemble as in [5]. In this work w 1 , w 2 = 0.5 are found optimal.…”

Section: Image and Text Ensemblementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Improving Accuracy and Speeding Up Document Image Classification Through Parallel Systems

Ferrando

Domínguez

Torres

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Работа [3] посвящена повышению эффективности обучения классификаторов на основе областей и их объединения для классификации изображений документов, метод достигает точности в 92,21 %. В [4] предложен подход, основанный на выделении, анализе и объединении текстового и визуального потоков для классификации изображений документов, в визуальном потоке используются глубокие CNN для извлечения структурных особенностей изображений, точность зависит от вида входных данных. В исследовании [5] предлагается двухпоточная нейронная архитектура для выполнения задачи классификации изображений документов, при этом используется подход совместного обучения признаков, объединяющий признаки изображения и текстовые части, подход совместного обучения имеет точность классификации до 97,05 %.…”

Section: Abstract: Document Management Automation Intelligent Document Management Document Classification Convolutional Neural Network Imunclassified

Classification of Scanned Documents Using a Convolutional Neural Network

Kotyuzhanskiy¹,

Chetverkin²,

Protasevich³

et al. 2021

СНТ (MHT)

View full text Add to dashboard Cite

В настоящее время одной из актуальных задач автоматизации документооборота организации в условиях поступления разнообразной документации от большого количества контрагентов является проверка и классификация сканированных материалов. В статье представлен анализ и основные характеристики существующих способов решения данной задачи. Целью исследования является разработка программного модуля, позволяющего классифицировать документы с точностью не менее 97 % в режиме реального времени, что актуально для электронного документооборота в крупных и средних компаниях. Приведено описание решения поставленной задачи на основе сверточной нейросети (CNN -Convolutional Neural Network). Входными данными для программного модуля является pdf-файл сканированного документа, выходными данными является xml-файл с классом документа. Для повышения точности и скорости работы программы были решены задачи по кодированию сигнала для нейронной сети и определению ее структуры. Приведено описание этапов обработки сканированных документов и архитектуры разработанной нейросети. Предложенный метод классификации позволяет классифицировать страницы с высокой точностью на небольшом датасете. Проведено тестирование программы на датасете из 9628 страниц и 22 возможных классов. Точность составила 99,1 %. Время классификации одной страницы без учета чтения файла и копирования в GPU составляет 2 мс на GeForce 780TI. Полное время классификации страницы составляет примерно 22,3 мс.Ключевые слова: автоматизация документооборота, интеллектуальный документооборот, классификация документов, сверточная нейросеть, распознавание изображений

show abstract

“…The primary reason for this is the striking difference between natural scenes and document images. This discrepancy has been noted in prior work, but in a limited context of comparing pretrained models as a good initialization point for the optimization process to establish their utility [13], [16], [18]. Furthermore, they assume the presence of large labeled document image datasets.…”

Section: Introductionmentioning

confidence: 96%

“…Therefore, this paper answers a fundamental question regarding the usefulness of natural scene image feature extractors to represent document images in contrast to documentspecific representations learned without any labeled data, which has been a common practice for document image classification [13], [16]- [18]. Specifically, the contributions of this paper are three-fold:…”

Section: Introductionmentioning

confidence: 99%

Self-Supervised Representation Learning for Document Image Classification

Siddiqui¹,

Dengel

Ahmed

2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

Supervised learning, despite being extremely effective, relies on expensive, time-consuming, and error-prone annotations. Self-supervised learning has recently emerged as a strong alternate to supervised learning in a range of different domains as collecting a large amount of unlabeled data can be achieved by simply crawling the internet. These self-supervised methods automatically discover features relevant to represent an input example by using self-defined proxy tasks. In this paper, we question the potential of commonly employed purely supervised training (starting either from ImageNet pretrained networks or pure random initialization) in contrast to self-supervised representations that can be learned directly using self-supervised representation learning methods on large document image datasets. For this purpose, we leverage a large-scale document image collection (RVL-CDIP) to train ResNet-50 image encoder using two different self-supervision methods (SimCLR and Barlow Twins). Employing a linear classifier on top of self-supervised embeddings from ResNet-50 results in an accuracy of 86.75% as compared to 71.43% from the corresponding ImageNet pretrained embeddings. Similarly, evaluating on Tobacco-3482 dataset using self-supervised embeddings from ResNet-50 yields an accuracy of 88.52% in contrast to 74.16% from the corresponding ImageNet pretrained embeddings. We show that in the case of limited labeled data, this wide gap in performance between self-supervised and fully supervised models persists even after finetuning pretrained models. However, a significant reduction in this gap is observed with an increasing amount of data including the case where the model is trained from scratch. Our results show that representations learned using self-supervised representation learning techniques are a viable option for document image classification, specifically in the context of limited labeled data, which is a usual restriction in industrial use cases.

show abstract

Two Stream Deep Network for Document Image Classification

Cited by 30 publications

References 25 publications

Improving Accuracy and Speeding Up Document Image Classification Through Parallel Systems

Improving Accuracy and Speeding Up Document Image Classification Through Parallel Systems

Classification of Scanned Documents Using a Convolutional Neural Network

Self-Supervised Representation Learning for Document Image Classification

Contact Info

Product

Resources

About