Identifying Malicious DNS Tunnel Tools from DoH Traffic Using Hierarchical Machine Learning Classification

Mitsuhashi, Rikima; Satoh, Akihiro; Jin, Yong; Iida, Katsuyoshi; Shinagawa, Takahiro; Takai, Yoshiaki

doi:10.1007/978-3-030-91356-4_13

Cited by 12 publications

(9 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our previous work [10], the implementation only recognized well-known malicious DNS tunnel tools, including dns2tcp, dnscat2, and iodine. However, there are a number of malicious DNS tunnel tools, such as ozymanDNS, DeNiSe, Heyoka, DNScapy, and many more.…”

Section: E Knowledge Updatementioning

confidence: 99%

“…• The evaluation results confirm that the proposed system is able to recognize the six malicious DNS tunnel tools with high classification accuracy. This article is an extended version of our previous publication [10]. In the previous paper, the proposed system was able to identify the well-known malicious DNS tunnel tools but could not recognize the newly emerged ones.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Malicious DNS Tunnel Tool Recognition Using Persistent DoH Traffic Analysis

Mitsuhashi

Jin

Iida

et al. 2023

IEEE Trans. Netw. Serv. Manage.

View full text Add to dashboard Cite

DNS over HTTPS (DoH) protocol can mitigate the risk of privacy breaches but makes it difficult to control network security services due to the DNS traffic encryption. However, since malicious DNS tunnel tools for the DoH protocol pose network security threats, network administrators need to recognize malicious communications even after the DNS traffic encryption has become widespread. In this paper, we propose a malicious DNS tunnel tool recognition system using persistent DoH traffic analysis based on machine learning. The proposed system can accomplish continuous knowledge updates for emerging malicious DNS tunnel tools on the machine learning model. The system is based on hierarchical machine learning classification and focuses on DoH traffic analysis. The evaluation results confirm that the proposed system is able to recognize the six malicious DNS tunnel tools in total, not only well-known ones, including dns2tcp, dnscat2, and iodine, but also the emerging ones such as dnstt, tcp-over-dns, and tuns with 98.02% classification accuracy.

show abstract

Section: E Knowledge Updatementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Malicious DNS Tunnel Tool Recognition Using Persistent DoH Traffic Analysis

Mitsuhashi

Jin

Iida

et al. 2023

IEEE Trans. Netw. Serv. Manage.

View full text Add to dashboard Cite

show abstract

“…According to the figure, the indicators of malicious activities include: Sudden spikes in DoH traffic: a sudden increase in network traffic could indicate malicious actors are using DoH to bypass DNS filters [ 14 ]. DoH traffic to suspicious domains: if a lot of DoH traffic goes to domains known to be associated with malware, phishing, or other types of malicious activity, it could be a sign that malicious actors are using DoH to access those domains [ 15 ]. Encrypted DoH traffic from known malware-infected hosts: if hosts on the network are known to be infected with malware, and encrypted DoH traffic is coming from those hosts, it could be a sign that the malware is using DoH to communicate with its command and control servers [ 16 ].…”

Section: Introductionmentioning

confidence: 99%

A Lightweight Double-Stage Scheme to Identify Malicious DNS over HTTPS Traffic Using a Hybrid Learning Approach

Al‐Haija

Alohaly

Odeh

2023

Sensors

View full text Add to dashboard Cite

The Domain Name System (DNS) protocol essentially translates domain names to IP addresses, enabling browsers to load and utilize Internet resources. Despite its major role, DNS is vulnerable to various security loopholes that attackers have continually abused. Therefore, delivering secure DNS traffic has become challenging since attackers use advanced and fast malicious information-stealing approaches. To overcome DNS vulnerabilities, the DNS over HTTPS (DoH) protocol was introduced to improve the security of the DNS protocol by encrypting the DNS traffic and communicating it over a covert network channel. This paper proposes a lightweight, double-stage scheme to identify malicious DoH traffic using a hybrid learning approach. The system comprises two layers. At the first layer, the traffic is examined using random fine trees (RF) and identified as DoH traffic or non-DoH traffic. At the second layer, the DoH traffic is further investigated using Adaboost trees (ADT) and identified as benign DoH or malicious DoH. Specifically, the proposed system is lightweight since it works with the least number of features (using only six out of thirty-three features) selected using principal component analysis (PCA) and minimizes the number of samples produced using a random under-sampling (RUS) approach. The experiential evaluation reported a high-performance system with a predictive accuracy of 99.4% and 100% and a predictive overhead of 0.83 µs and 2.27 µs for layer one and layer two, respectively. Hence, the reported results are superior and surpass existing models, given that our proposed model uses only 18% of the feature set and 17% of the sample set, distributed in balanced classes.

show abstract

“…According to [6], [7] on the use of DNS TXT resource records, some types of bot programs have been identified as using DNS TXT resource records for botnet communication. In [8], [9], the authors proposed a machine-learning-based detection method of botnet communications using DNS over HTTPS protocol, which is a privacy enhancement of DNS. As a result, the DNS traffic which so far has been considered secure also has become a target of being monitored communication since the network administrators cannot simply block all DNS traffic.…”

Section: Introductionmentioning

confidence: 99%

Policy-based Detection and Blocking System for Abnormal Direct Outbound DNS Queries using RPZ

2022

Proceedings of 2022 the 12th International Workshop on Computer Science and Engineering

View full text Add to dashboard Cite

Bot-infected computers sending direct outbound DNS queries without obtaining the information of authoritative DNS servers from the DNS full resolvers set up in the internal network have become a critical security issue nowadays. In DNS protocol, the domain name resolution process obtains the information of necessary authoritative DNS name servers (NS records) at the beginning and then achieves the answers of the original DNS queries which is accomplished via the DNS full-service resolvers. However, some types of bot programs violate the DNS protocol process and send the direct outbound DNS queries to its Command and Control (C&C) servers (malicious DNS servers) for bot communication. We have investigated the detection and blocking the direct outbound DNS queries by using MySQL at an early stage. However, the network latency was arising as a critical issue. In this advanced research, we propose a policybased detection and blocking system for abnormal direct outbound DNS queries using DNS Response Policy Zones (DNS RPZ) in order to solve the issues. In this paper, we describe the design of the proposed system and introduce an implemented prototype system. In addition, we also describe the preliminary evaluation results per feature of the proposed system conducted on the prototype, and finally, we introduce the tasks planed for future work.

show abstract

Identifying Malicious DNS Tunnel Tools from DoH Traffic Using Hierarchical Machine Learning Classification

Cited by 12 publications

References 18 publications

Malicious DNS Tunnel Tool Recognition Using Persistent DoH Traffic Analysis

Malicious DNS Tunnel Tool Recognition Using Persistent DoH Traffic Analysis

A Lightweight Double-Stage Scheme to Identify Malicious DNS over HTTPS Traffic Using a Hybrid Learning Approach

Policy-based Detection and Blocking System for Abnormal Direct Outbound DNS Queries using RPZ

Contact Info

Product

Resources

About