Regular Expression Software Deceleration for Intrusion Detection Systems

Abstract

The use of reconfigurable hardware for network security applications has recently made great strides as FPGA devices have provided larger and faster resources. Regular expressions have become a necessary and basic capability of Intrusion Detection Systems, but their implementation tends to be expensive in terms of memory cost and time performance. This work provides an architecture that reduces the exponential NFA to DFA conversion cost to a linear growth for many expressions.By handling the timing and integration of the regular expression-based rules in a custom microcontroller, the memory costs are reduced and the capabilities are increased over a DFA-only solution. Both the microcontroller and its associated DFA are implemented on the FPGA. The patterns and software are stored using run-time programmable memory tables. This allows on-the-fly modification to the regular expressions.This paper presents the design details of the regular expression microcontroller and its integration to the DFA state machines. The types of expressions that the system can handle efficiently are discussed as well as the outstanding problems that continue to challenge the community.

A Dynamically Reconfigurable FPGA-Based Pattern Matching Hardware for Subclasses of Regular Expressions

“…A recent research trend in large-scale regular expression matching hardwares is to simulate finite state automata for a class of regular expressions on a specially designed hardware [3], [4], [8], [13], [19], [20], [24]- [26]. This approach is further classified into the static compilation approach and the dynamic reconfiguration approach.…”

“…In the dynamic reconfiguration approach [3], [4], [8], [13], a universal control logic is statically compiled into FPGA beforehand as well, but a description of regular expressions is dynamically loaded to the FPGA as data in preprocessing, and then simulated in run-time. This approach is attractive in real-world applications such as ESP and NIDS, where input patterns frequently change.…”

“…Overall, our research goal is to design dynamically re- configurable hardwares for complex classes of regular expressions and to achieve gigabit throughput as the hardwares by [3], [4], [6], [8], [14], [21].…”

“…Event stream processing (ESP) [1] and network intrusion detection system (NIDS) [3] are example applications of data stream processing. Consequently, efficient data stream processing technologies have been extensively studied in theory and practice.…”

“…The large-scale pattern matching problem [3], [5], [13], [15], [20], [24], [26] is one of the most important problems in data stream processing, where a pattern matching system has to work with a large number (e.g., from thousands to several tens thousands) of complex patterns (e.g., regular expressions) against high-speed data streams (e.g., of several giga bps). For example, large-scale regular expression matching is such a problem that appears in realworld applications such as ESP and NIDS, where regular expressions are defined by characters, concatenation, union, and Kleene-star.…”