SA Sketch: A self‐adaption sketch framework for high‐speed network

Abstract

Sketch is a compact data structure used to summarize data streams. It is widely used in the measurement of network traffic, and its accuracy is higher than traditional methods. Currently, there are some typical sketches: Count-Min Sketch, CU Sketch, and Count Sketch. According to the characteristics of network traffic, we propose a new sketch framework called Self-Adaption Sketch, which is combined Sketch with Bloom Filter. In the framework, the sketch is created dynamically and the memory space is adjusted timely according to the network traffic by using the concept carrying. Our experiment results showed that the space utilization and accuracy are significantly improved while the throughput of self-adaption sketch is maintained at a relatively good level. K E Y W O R D S traffic measurement, data stream technology, sketch, bloom filter, space utilization 1 INTRODUCTION 1.1 Background and motivation Network measurement and analysis are essential for network management and they help network administrators to understand the status of the network better and make the right decisions. They help to detect anomalies and attacks in the network. To provide a secure and well-performing network for the continually transforming cyberspace, Internet operators need to monitor and analyze the network status in real time. 1 In recent years, network traffic measurement has been widely used in network accounting, traffic engineering, network security, and other fields. However, there are certain difficulties in the measurement, which are mainly due to the continuous and massive network traffic. At present, there are two main types of high-speed network traffic measurement schemes, one is the sampling technology, and the other is the data stream technology. Sampling technology refers to the collection and processing of some representative network traffic. This method introduces the small system load with a relatively large error. Typical sampling methods are group sampling (such as BSS, 2 EBS 3) and stream sampling (such as ANF 4). Collapsed Gibbs sampling is the most commonly used algorithm that samples latent topics for a word occurrence (token) by integrating out the Dirichlet priors. 5 The sampling technique has been widely used, such as quantile calculation, data collection, and top-k query. 6 Data stream technology processes

Single-Instruction-Multiple-Data Instruction-Set-Based Heat Ranking Optimization for Massive Network Flow

“…If each packet is allocated a counter to store, although the measurement is accurate, the storage of all data flows and the need to search all the historical data flows will require a lot of memory space because of the huge amount of data in flow mining. Therefore, based on the hash value, the required storage space is determined according to the range of hash values [20]. Various packages are reclassified according to the hash value, and the data stream is compressed and stored in a smaller space, which can greatly reduce the storage space [21].…”

Emerging intelligent big data analytics for cloud and edge computing

“…Currently, there are some typical sketches: Count-Min Sketch, CU Sketch, and Count Sketch. According to the characteristics of network traffic, Zhu et al 10 propose a new sketch framework called Self-Adaption Sketch, which combines Sketch and Bloom Filter.…”

“…Currently, there are some typical sketches: Count‐Min Sketch, CU Sketch, and Count Sketch. According to the characteristics of network traffic, Zhu et al 10 propose a new sketch framework called Self‐Adaption Sketch, which combines Sketch and Bloom Filter. In the framework, the sketch is created dynamically and the memory space is adjusted timely according to the network traffic by using the concept carrying.…”