Document Image Classification Method Based on Graph Convolutional Network

Xiong, Yangyang; Dai, Zhongjian; Liu, Yan; Ding, Xiaotian

doi:10.1007/978-3-030-92185-9_26

Cited by 8 publications

(12 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As can be seen from the table, our best performing model DocXClassifier-XL achieved 94.17% accuracy on the RVL-CDIP dataset, outperforming all previous image-based methods by a significant margin of +1.86%. It is interesting to note that even our lightest variant DocXClassifier-B achieved a comparable accuracy of 94.00%, and performed significantly better than all existing image-based models as well as some of the more sophisticated multimodal approaches [35], [46], [47], thus representing a good trade-off between accuracy and computational cost. It is important to note that two of the best performing multimodal solutions, those of Kanchi et al (2022) [48] and Bakkali et al (2020) [17], simply combined ConvNetbased visual backbones (EfficientNet and NasNet, respectively) with a Transformer-based textual backbone (BERT) to achieve extraordinary improvements in document classification.…”

Section: B Overall Evaluationmentioning

confidence: 92%

“…Recently, there has been an increased emphasis on multimodal classification techniques [13], [34], [35], in which document images are preprocessed to extract the textual content using stand-alone OCR software, and then visual, textual, and other layout features are used together for classification. Initial work in this area focused mainly on generating textual and visual embeddings using two separate deep network streams [12], [13] and then integrating them into a single embedding for final classification.…”

Section: Related Work a Document Image Classificationmentioning

confidence: 99%

“…However, both approaches of these approaches require pre-training with large amounts of document data. In a slightly different direction, Graph ConvNets [35] have also been recently explored for multimodal classification and show promising results.…”

Section: Related Work a Document Image Classificationmentioning

confidence: 99%

See 2 more Smart Citations

DocXClassifier: Towards an Interpretable Deep Convolutional Neural Network for Document Image Classification

Saifullah¹,

Agne²,

Dengel³

et al. 2022

Preprint

View full text Add to dashboard Cite

<p> Convolutional Neural Networks (ConvNets) have been thoroughly researched for document image classification and are known for their exceptional performance in unimodal image-based document classification. Recently, however, there has been a sudden shift in the field towards multimodal approaches that simultaneously learn from the visual and textual features of the documents. While this has led to significant advances in the field, it has also led to a waning interest in improving pure ConvNets-based approaches. This is not desirable, as many of the multimodal approaches still use ConvNets as their visual backbone, and thus improving ConvNets is essential to improving these approaches. In this paper, we present DocXClassifier, a ConvNet-based approach that, using state-of-the-art model design patterns together with modern data augmentation and training strategies, not only achieves significant performance improvements in image-based document classification, but also outperforms some of the recently proposed multimodal approaches. Moreover, DocXClassifier is capable of generating transformer-like attention maps, which makes it inherently interpretable, a property not found in previous image-based classification models. Our approach achieves a new peak performance in image-based classification on two popular document datasets, namely RVL-CDIP and Tobacco3482, with a top-1 classification accuracy of 94.17% and 95.57% on the two datasets, respectively. Moreover, it sets a new record for the highest image-based classification accuracy of 90.14% on Tobacco3482 without transfer learning from RVL-CDIP. Finally, our proposed model may serve as a powerful visual backbone for future multimodal approaches, by providing much richer visual features than existing counterparts. </p>

show abstract

Section: B Overall Evaluationmentioning

confidence: 92%

Section: Related Work a Document Image Classificationmentioning

confidence: 99%

See 1 more Smart Citation

DocXClassifier: Towards an Interpretable Deep Convolutional Neural Network for Document Image Classification

Saifullah¹,

Agne²,

Dengel³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently, there has been an increased emphasis on multimodal classification techniques [15], [35], [36], in which document images are preprocessed to extract the textual content using stand-alone OCR software, and then visual, textual, and other layout features are used together for classification. Initial work in this area focused mainly on generating textual and visual embeddings using two separate deep network streams [14], [15] and then integrating them into a single embedding for final classification.…”

Section: Related Work a Document Image Classificationmentioning

confidence: 99%

“…However, both approaches require pre-training with large amounts of document data. In a slightly different direction, Graph ConvNets [36] have also been recently explored for multimodal classification and show promising results.…”

Section: Related Work a Document Image Classificationmentioning

confidence: 99%