A High Throughput String Matching Architecture for Intrusion Detection and Prevention

Abstract

Network Intrusion Detection and Prevention Systems have emerged as one of the most effective ways of providing security to those connected to the network, and at the heart of almost every modern intrusion detection system is a string matching algorithm. String matching is one of the most critical elements because it allows for the system to make decisions based not just on the headers, but the actual content flowing through the network. Unfortunately, checking every byte of every packet to see if it matches one of a set of ten thousand strings becomes a computationally intensive task as network speeds grow into the tens, and eventually hundreds, of gigabits/second. To keep up with these speeds a specialized device is required, one that can maintain tight bounds on worst case performance, that can be updated with new rules without interrupting operation, and one that is efficient enough that it could be included on chip with existing network chips or even into wireless devices. We have developed an approach that relies on a special purpose architecture that executes novel string matching algorithms specially optimized for implementation in our design. We show how the problem can be solved by converting the large database of strings into many tiny state machines, each of which searches for a portion of the rules and a portion of the bits of each rule. Through the careful co-design and optimization of our our architecture with a new string matching algorithm we show that it is possible to build a system that is 10 times more efficient than the currently best known approaches.

Efficient hardware support for pattern matching in network intrusion detection

“…Since Cho's [2] is an ASIC implementation, it has a large clock frequency at the cost of rigidity to updates. Also, another ASIC solution in [18] involves memory tiles where a 2-bit input selects one of four finite state machines. Although [18] does not list the number of patterns in the implementation, it contains a comparison with the design proposed in [11] that assumes 1466 rules with 18,031 characters.…”

“…Also, another ASIC solution in [18] involves memory tiles where a 2-bit input selects one of four finite state machines. Although [18] does not list the number of patterns in the implementation, it contains a comparison with the design proposed in [11] that assumes 1466 rules with 18,031 characters. The work in [18] uses 3200 Kbits of memory, yielding a memory consumption ratio of 181.7 bits per character which is quite high compared to our design (see Table II).…”

“…Although [18] does not list the number of patterns in the implementation, it contains a comparison with the design proposed in [11] that assumes 1466 rules with 18,031 characters. The work in [18] uses 3200 Kbits of memory, yielding a memory consumption ratio of 181.7 bits per character which is quite high compared to our design (see Table II). Also, our updating process is very simple as it does not require intricate knowledge of our design.…”

Multi-Character Processor Array for Pattern Matching in Network Intrusion Detection System

“…A large number of comparisons are required for dealing with multiple characters, as shown in Figure 1, because all possible combinations of characters are compared against the input string in parallel. The deterministic finite automata (DFA) approach [2,7,10] uses a state machine to track partial pattern matches across clock cycles. A DFA will take in a string of input character.…”

An architecture for exploiting multi-core processors to parallelize network intrusion prevention

“…FPGA-based work in this regard has investigated the use of non-deterministic finite automata to match regular expressions [28,29], compiling regular expressions into deterministic finite automata [30], and then quickly generating new, compiled FPGA binaries [31]. Other custom hardware efforts, not specific to FPGAs, have investigated building optimized Aho-Corasick trees for sets of strings [32] and specialized architectures based on collections of highly optimized tiny state machines, each of which looks for a portion of a string [33].…”