A Prefiltering Approach to Regular Expression Matching for Network Security Systems

Abstract

Abstract. Regular expression (RegEx) matching has been widely used in various networking and security applications. Despite much effort on this important problem, it remains a fundamentally difficult problem. DFA-based solutions can achieve high throughput, but require too much memory to be executed in high speed SRAM. NFA-based solutions require small memory, but are too slow. In this paper, we propose RegexFilter, a prefiltering approach. The basic idea is to generate the RegEx print of RegEx set and use it to prefilter out most unmatched items. There are two key technical challenges: the generation of RegEx print and the matching process of RegEx print. The generation of RegEx is tricky as we need to tradeoff between two conflicting goals: filtering effectiveness, which means that we want the RegEx print to filter out as many unmatched items as possible, and matching speed, which means that we want the matching speed of the RegEx print as high as possible. To address the first challenge, we propose some measurement tools for RegEx complexity and filtering effectiveness, and use it to guide the generation of RegEx print. To address the second challenge, we propose a fast RegEx print matching solution using Ternary Content Addressable Memory. We implemented our approach and conducted experiments on real world data sets. Our experimental results show that RegexFilter can speedup the potential throughput of RegEx matching by 21.5 times and 20.3 times for RegEx sets of Snort and L7-Filter systems, at the cost of less than 0.2 Mb TCAM chip.

Moore-Machine Filtering for Timed and Untimed Pattern Matching

“…Efficiency of pattern matching has been actively pursued in the fields of database [17], [18] and networking [12], [19], [26]- [29]. In these fields, issues in hardware architecturesuch as speed gap between L1/L2 caches and main memoryconstitute situations similar to that of embedded monitors that we discussed in the above.…”

“…In database and networking, pattern matching against regular expressions is mainly approached by application-specific heuristics that often take machine architecture into account. For example, in [19], [26] pattern regular expressions undergo application of some application-specific rewrite rules. The hierarchical structure of L1/L2 caches and main memory is exploited in [17], [18], while the features of content-addressable memory (CAM)-an alternative to RAM often used in network devices-are exploited in [12].…”

“…Pre-filtering before actual pattern matching has been considered in the above lines of works [17], [26], [32]. The principal difference between those works and ours is that their filters produce match candidates, data that include explicit indices for potential matches.…”

Based on Regular Expression Matching of Evaluation of the Task Performance in WSN: A Queue Theory Approach

“…However, even if techniques such as D2FA [13, 14] are employed, tables of DFA and NFA are too big to be stored in TCAMs. In 2012, the RegexFilter (a high-speed and memory efficient technique) was presented by Liu et al [15]. Regular expression matching was been sped up by quickly searching these regular expressions that may match each arriving item as little as possible.…”

“…An item that does not match any regular expression in the set [15] is unmatched item of a regular expression set. Given an item i , it will match against R ′ to get set O ( R ′; i ) firstly.…”