Hadoop‐based analytic framework for cyber forensics

Abstract

With an exponential increase in the data size and complexity of various documents to be investigated, existing methods of network forensics are found not much efficient with respect to accuracy and detection ratio. The existing techniques for network forensic analysis exhibit inherent limitations while processing a huge volume, variety, and velocity of data. It makes network forensic a time-consuming and resource-consuming task. To balance time taken and output delivered, these existing techniques put a limit on the amount of data under analysis, which results in a polynomial time complexity of these solutions. So to mitigate these issues, in this paper, we propose an effective framework to overcome the limitation to handle large volume, variety, and velocity of data. An architectural setup that consists of MapReduce framework on top of Hadoop Distributed File System environment is proposed in this paper. The proposed framework demonstrates its capability to handle issues of storage and processing of big data using cloud computing. Also, in the proposed framework, supervised machine learning (random forest-based decision tree) algorithm has been implemented to demonstrate better sensitivity. To train and validate the model, online available data set from CAIDA is taken and university network traffic samples, with increasing size, has been taken for experiment. Results thus obtained confirm the superiority of the proposed framework in network forensics, with an average accuracy of 99.34% (malicious and nonmalicious traffic).

Distributed frameworks for detecting distributed denial of service attacks: A comprehensive review, challenges and future directions

“…Sachdeva et al 30 recommended the distributed internet service provider (ISP) domain DDoS deployment because of several reasons: (i) Distributed environment, (ii) Practically feasible solution, (iii) Robust implementation, (v) Autonomous control, etc. Further, few researchers 31‐43 proposed distributed processing framework‐based DDoS detection systems. This type of deployment not only quickly recognizes attacks but also efficiently explores a huge volume of traffic by employing a cluster of slaves in a coordinated manner.…”

“…References 31‐42, proposed Hadoop‐ and Spark‐based distributed DDoS attack detection mechanisms. A victim‐end deployment is one of the best choice to implement distributed processing frameworks‐based mechanisms because of several reasons:It is deployed near to the victim system or in victim networks.…”

“…Therefore, this type of framework is not suitable for source‐end deployment. In traditional framework‐based victim‐end DDoS detection received a massive (aggregated) amount of network traffic to analyze, and sometimes systems itself becomes a victim of an attack while analyzing such massive traffic. Therefore, one of the best distributed approach is to implement a victim‐end DDoS mechanisms and most of researchers 31‐42 have used this type deployment.…”

“…Few authors 31‐43,91,92,95,96 proposed distributed framework‐based mechanisms and most of them deployed their mechanism on the Apache Hadoop. Apache Hadoop distributed framework follows a two‐step process to analyze data efficiently.…”

“…Alsirhani et al 39 proposed a DDoS detection system similar to Reference 38 and applied Gradient Boosting to identify malicious traffic. However, both systems 38,39 required more time. Chhabra et al 42 proposed a victim‐end DDoS forensics analytics technique deployed on the Apache Hadoop distributed framework. They employed a supervised random forest‐based decision tree algorithm to design a detection model and claimed an approximate 99% accuracy.…”