A Framework for Understanding Unstructured Financial Documents Using RPA and Multimodal Approach

Abstract

The financial business process worldwide suffers from huge dependencies upon labor and written documents, thus making it tedious and time-consuming. In order to solve this problem, traditional robotic process automation (RPA) has recently been developed into a hyper-automation solution by combining computer vision (CV) and natural language processing (NLP) methods. These solutions are capable of image analysis, such as key information extraction and document classification. However, they could improve on text-rich document images and require much training data for processing multilingual documents. This study proposes a multimodal approach-based intelligent document processing framework that combines a pre-trained deep learning model with traditional RPA used in banks to automate business processes from real-world financial document images. The proposed framework can perform classification and key information extraction on a small amount of training data and analyze multilingual documents. In order to evaluate the effectiveness of the proposed framework, extensive experiments were conducted using Korean financial document images. The experimental results show the superiority of the multimodal approach for understanding financial documents and demonstrate that adequate labeling can improve performance by up to about 15%.

Information Extraction from Financial Statements based on Visually Rich Document Models

“…Em [Cho et al 2023] tem-se a proposta de um framework IDP que combina o uso do um modelo de aprendizado profundo pré-treinado LayoutXLM com técnicas de RPA (automac ¸ão de processos robóticos), para extrair informac ¸ões do tipo chave-valor em documentos financeiros coreanos e capturar dados em sistemas externos. O uso do LayoutXLM visou flexibilizar a aplicac ¸ão do Framework em vários tipos de documentos.…”

Design and practice of automatic analysis system for economic operation mode of power grid

“…In order to solve this problem, this paper proposes a fully automatic intelligent analysis system for power grid economic operation mode based on RPA+LLM. This system uses Robotic Process Automation (RPA) technology to automatically capture and integrate power grid operation data [8], and uses Large Language Models (LLM) technology to automatically generate text for power grid economic operation mode analysis [9][10]. The system has the characteristics of fast data acquisition, accurate data processing, automatic generation of analysis reports, etc., providing effective support and reference for optimizing the economic operation mode of the power grid.…”