Tunnel Security Management Based on Association Rule Mining under Hadoop Platform

Abstract

The problem of how to use large amounts of historical data for tunnel safety management has a greater practical application value. The association rule method in data mining technology can provide effective decision support for tunnel safety prevention by mining historical data. To address the problem of large data volume and sparse data items in tunnel safety management, an association rule method—Apriori algorithm—based on the Hadoop platform is proposed to improve the efficiency and accuracy of data mining in cloud environment. First, the parallel MapReduce implementation steps are analyzed on the basis of the distributed Hadoop framework. Then, the existing single-user data validation algorithm is improved by applying a multiuser parallel validation algorithm to Apriori in order to reduce the number of validations. Next, the traditional association rule Apriori algorithm is MapReduce optimized to generate a smaller set of useless candidate items. At the same time, Boolean ranking is used to optimize the way transactional data are stored in the database, reducing the number of redundant subsets and the number of times the database is connected, and shortening the task processing time. The experimental results show that the proposed method is able to mine the relationships between tunnel safety hazards and provide effective decision support for tunnel safety prevention. At the same time, the proposed method more efficiently operates than other association rule methods.