A Framework for Detecting Botnet Command and Control Communication over an Encrypted Channel

Abstract

Botnet employs advanced evasion techniques to avoid detection. One of the Botnet evasion techniques is by hiding their command and control communication over an encrypted channel like SSL and TLS. This paper provides a Botnet Analysis and Detection System (BADS) framework for detecting Botnet. The BADS framework has been used as a guideline to devise the methodology, and we divided this methodology into six phases: i. data collection, customization, and conversion, ii. feature extraction and feature selection, iii. Botnet prediction and classification, iv. Botnet detection, v. attack notification, and vi. testing and evaluation. We tend to use the machine learning algorithm for Botnet prediction and classification. We also found several challenges in implementing this work. This research aims to detect Botnet over an encrypted channel with high accuracy, fast detection time, and provides autonomous management to the network manager.

Reviewing various feature selection techniques in machine learning‐based botnet detection

“…Experimental results shows that KNN classification algorithm yields more accurate results than any other classification algorithm, regardless of the FS techniques used. Ismail et al 32 have proposed a P2P botnet detection approach using various ML algorithms. They have also used feature extraction and FS.…”

Botnet Detection and Incident Response in Security Operation Center (SOC): A Proposed Framework

“…Based on literature [14], there are three major methods of botnet detection such as host-based detection, honeynet detection and network-based detection. Recently, machine learning based detection has become the most widely used for detecting botnets methods as proven by previous literature [15], [16], [4], [5]. In addition, the number and complexity of IoT devices is also growing, it has become important to develop effective botnet detection methods.…”

“…Ismail et al [4] proposed a Botnet Analysis and Detection System (BADS) which could detect Botnet in encrypted channel and includes the autonomous feature. The BADS framework comprises of three main components which are Network Analysis System (NAS), IDS and Alarm System www.ijacsa.thesai.org (AS).…”

“…Once infected, the botmaster will gain access to the victim's computer without victim acknowledgment [3]. In [4], botnet can perform DDoS attacks, spamming, malware and compromise a large computer. These activities pose a significant risk to national security, public or private organizations, and individual.…”