Web Application Firewall Using Machine Learning and Features Engineering

Abstract

Web application security has become a major requirement for any business, especially with the wide web attacks spreading despite the defensive measures and the continuous development of software frameworks and servers. In this study, we present a proposed model for a web application firewall that used machine learning and features engineering to detect common web attacks. Our proposed model analyses incoming requests to the webserver, parses these requests to extract four features that describe completely HTTP request parts (URL, payload, and headers), and classifies whether a request is normal or an anomaly. We took into consideration the limitation of previous works that use URL and payload only in classification and provided five features that describe and summarize all parts of the HTTP request using features engineering and previous experience in the field of the software security domain. Extracted features are length of request, percentage of characters allowed, percentage of special characters, and attack weight. These features were calculated for four different datasets CSIC 2010, HTTPParams 2015, Hybrid dataset (CSIC 2010 and HTTPParams), and real logs for the compromised web server. We evaluated our proposed model by using these updated datasets with four classification algorithms (Naive Bayes, logistic regression, decision tree, and support vector machine) with two methods (train test split and cross-validation) to negate the probability of overfitting and ensure that features are effective. Features values for a normal request are usually short request length, large allowed character ratio, small special character ratio, and zero attack weight or close to zero. Features values for anomaly requests are large request length, small allowed character percentage, large special character percentage, and very large numerically attack weight. Our proposed model achieved a classification accuracy of 99.6% with datasets used in research studies in this field and 98.8% with datasets of real web servers.

Neural Network-Based Web Security Firewall

“…The proposed model analyzes incoming requests to the web server, parses these requests to extract four features that describe completely HTTP request parts (URL, payload, and headers), and classifies whether a request is normal or an anomaly. The model achieved a classification accuracy of 99.6% with datasets used in research studies in this field and 98.8% with datasets of real web servers [3].…”

“…CSIC: This dataset contains HTTP traffic data collected from a real web application. It includes both normal and malicious traffic and has been widely used in previous research studies[3].3. HTTP Params: This dataset contains HTTP traffic data collected from a real web application.…”

Model Design of Intrusion Detection System on Web Server Using Machine Learning Based

“…Machine Learning (ML) has revolutionized a myriad of sectors in the contemporary digital ecosystem, from healthcare to finance, and its importance cannot be understated when discussing cybersecurity and intrusion detection. At its core, machine learning is a subset of artificial intelligence that employs algorithms allowing computers to learn and make decisions or predictions from data without explicit programming [25], [26].…”

Mitigating Security Risks in Firewalls and Web Applications using Vulnerability Assessment and Penetration Testing (VAPT)

“…Validated through a Web Application Filter (WAF) use case, RiAS showcases the potential of adaptive, risk-based security measures to respond dynamically to threats, underscoring its relevance in modern, heterogeneous computing contexts. Shaheed et al [24] presents an advanced web application firewall model leveraging machine learning and feature engineering to detect web attacks. This model uniquely analyzes entire HTTP requests, including URL, payload, and headers, by extracting four key features: request length, percentages of allowed and special characters, and an attack weight.…”