Implementation of Parallel Lanczos Method for Intrusion Detection with Cloud Technologies

Abstract

The aim of dimensionality reduction is to construct a low-dimensional representation of high dimensional input data in such a way, that important parts of the structure of the input data are preserved. This paper proposes to apply the dimensionality reduction to intrusion detection data based on the parallel Lanczos-SVD (PLSVD) with the cloud technologies. The massive input data is stored on distribution files system, like HDFS. And the Map/Reduce method is used for the parallel analysis on many cluster nodes. Our experiment results show that, compared with the PCA algorithm, PLSVD algorithm has better scalability and flexibility.