Enhancing Detection of Malicious URLs Using Boosting and Lexical Features

Abstract

A malicious URL is a link that is created to spread spams, phishing, malware, ransomware, spyware, etc. A user may download malware that can adversely affect the computer by clicking on an infected URL, or might be convinced to provide confidential information to a fraudulent website causing serious losses. These threats must be identified and handled in a decent time and in an effective way. Detection is traditionally done through the blacklist usage method, which relies on keyword matching with previously known malicious domain names stored in a repository. This method is fast and easy to implement, with the advantage of having low false-positive rates regarding previously recognized malicious URLs. However, this method cannot recognize newly created malicious URLs. To solve this problem, many machine-learning models have been used. In this paper, we introduce an effective machine learning approach that uses an ensemble learner algorithm called AdaBoost (Adaptive Boosting), combined with different algorithms that enhance detection. For datasets filtration, we used CfsSubsetEval technique, which is an algorithm that searches for a subset of features that work well together. Datasets were collected from the UNB repository; divided into four categories: spam, phishing, malware, and defacement URLs; combined with benign URLs, dataset content is based on lexical features. The experimental results indicate that the proposed approach was successful in enhancing the detection accuracy of malicious URLs with less false-positive rates for all experimental algorithms.

An Assessment of Lexical, Network, and Content-Based Features for Detecting Malicious URLs Using Machine Learning and Deep Learning Models

“… Exec, Search, and Link functions: we searched for the presence of these further suspicious functions used by attackers in JavaScript code in the content features and stored its count in new features as numerical attributes. Total count of the suspicious functions: since all the above-mentioned functions are used by attackers for cross-site scripting and malware distribution [ 12 ], we added up the counts of all these functions that were present in the JavaScript code in the content features and stored its final count in a new feature called count_all_functions as a numerical attribute. Presence of Windows.open function: the Windows.open functions are generally used for ads popups and can be used to inject attacks on malicious sites.…”

“…Total count of the suspicious functions: since all the above-mentioned functions are used by attackers for cross-site scripting and malware distribution [ 12 ], we added up the counts of all these functions that were present in the JavaScript code in the content features and stored its final count in a new feature called count_all_functions as a numerical attribute.…”

“…Some researchers used only lexical features in their studies to assess intelligent models. Atrees et al [ 12 ] used the chi-square and ANOVA F-value to select the most important lexical features from various datasets. The researchers' goal was to discover the most significant features of URLs.…”