The use of Static Analysis Tools (SATs) is mandatory when developing secure software and searching for vulnerabilities in legacy software. However, the performance of the various SATs concerning the detection of vulnerabilities and false alarm rate is usually unknown and depends on many factors. The simultaneous use of several tools should increase the detection capabilities, but also the number of false alarms. In this paper, we study the problem of combining several SATs to best meet the developer needs. We present results of analyzing the performance of diverse static analysis tools, based on a previously published dataset that resulted from the use of five diverse SATs to find two types of vulnerabilities, namely SQL Injections (SQLi) and Cross-Site Scripting (XSS), in 132 plugins of the WordPress Content Management System (CMS). We present the results based on well-established measures for binary classifiers, namely sensitivity and specificity for all possible diverse combinations that can be constructed using these 5 SAT tools. We then provide empirically supported guidance on which combinations of SAT tools provide the most benefits for detecting vulnerabilities with low false positive rates.

Abstract-Defence-in-depth is a term often used in security literature to denote architectures in which multiple security protection systems are deployed to defend the valuable assets of an organization (e.g. the data and the services). In this paper we present an approach for analysing defence-in-depth, and illustrate the use of the approach with an empirical study in which we have assessed the detection capabilities of intrusion detection systems when deployed in diverse, two-version, parallel defence-in-depth configurations. The configurations have been assessed in settings that favour detection of attacks (reducing false negatives), as well as settings that favour legitimate traffic (reducing false positives).

This is the accepted version of the paper.This version of the publication may differ from the final published version. Abstract -In this paper we report results of an empirical analysis of the detection capabilities of 9 AntiVirus (AV) products when they were subjected to 3605 malware samples collected on an experimental network over a period of 31 days in NovemberDecember 2013. We compared the detection capabilities of the version of the AV products that the vendors make available for free in VirusTotal versus the full capability products that they make available via their own website. The analysis has been done using externally observable properties of the AV products: namely whether they detect a given malware. The paper reports extensive analysis of the results. A surprising finding of our study was that only one of the vendors had a full capability version which detected all the malware that their VirusTotal version could detect. Permanent repository link

We present AVAMAT: AntiVirus and Malware Analysis Tool-a tool for analysing the malware detection capabilities of AntiVirus (AV) products running on different operating system (OS) platforms. Even though similar tools are available, such as VirusTotal and MetaDefender, they have several limitations, which motivated the creation of our own tool. With AVAMAT we are able to analyse not only whether an AV detects a malware, but also at what stage of inspection does it detect it and on what OS. AVAMAT enables experimental campaigns to answer various research questions, ranging from the detection capabilities of AVs on OSs, to optimal ways in which AVs could be combined to improve malware detection capabilities.