Distributed Infrastructure for Big Data Processing in the Transaction Monitoring Systems

Abstract

To increase the effectiveness of detecting fraudulent bank transactions, the structure of the system is proposed to analyze data of user environment in order to identify potential fraudulent activities. The system for collecting and analyzing information about the user environment allows to accumulate data about the user environment, to mark precedents in manual and automatic modes and build a database of images for classifiers training. It is necessary to implement data collection, storage and access interface for the application of data mining tools. Operation of significant amount of accumulated data requires the use of special tools (frameworks and hardware platforms) for processing large data. In this paper the analysis of the existing software and hardware tools for distributed processing of indefinitely structured data of bank transactions (frameworks: Hadoop, Apache Spark) is presented. The structure and recommendations for the deployment of a hardware and software stand for testing algorithms for detecting financial fraud on the basis of data mining analysis as part of a distributed data processing system for bank transactions based on the selected framework are developed.