A comparative analysis about similarity search strategies for digital forensics investigations

Abstract

Known File Filtering method separates relevant from non-relevant information in forensics investigations using white or black lists. Due to limitations on hash functions (inability to detect similar data), approximate matching tools have gained focus recently. However, comparing two sets of approximate matching digests using brute force can be too time-consuming. Strategies to efficiently perform lookups in digests databases have been proposed as a form of similarity search. In this paper, we compare some strategies based on ssdeep and sdhash tools concerning to precision, memory requirement, and lookup complexity. We show that none of these strategies address these requirements satisfactorily.

An operational costs analysis of similarity digest search strategies using approximate matching tools

“…These new methods referred as similarity digest search strategies, aim at performing digest lookups very efficiently for KFF. This paper extends the work done in [Moia and Henriques 2017], where the authors performed the analysis of three different strategies (F2S2, MRSH-NET, and BF-based Tree) regarding three main aspects: precision, memory requirement, and lookup complexity. We show how the same strategies carry out the search and develop equations to estimate their operational costs, allowing a more precise time comparison.…”

“…Another important fact to mention is that even though F2S2 presents higher lookup complexity than MRSH-NET and BF-based tree [Moia and Henriques 2017], it performed better in our experiments when comparing large data sets. As expected, MRSH-NET was better than the BF-based tree.…”

“…However, this approach only exists in theory and lacks a working prototype. Also, it demands an enormous amount of memory to operate in comparison to other strategies [Moia and Henriques 2017].…”

“…Before calculating the operational costs, we determined the strategies structure size. To this end, we used the formulas and parameters presented in another work [Moia and Henriques 2017], adapting only the database and average object sizes. We measured the time to load an object of 1 GiB from disk to memory (buffer of s buf bytes) and adjusted this value according to the strategies structure size to simulate the loading.…”

A Digital Media Similarity Measure for Triage of Digital Forensic Evidence

“…Moia and Henriques [17] have presented steps for developing new approximate matching functions. Approximate matching functions overcome the limitations of cryptographic hash functions that cannot detect non-identical, but similar, data.…”