KDE-Track: An Efficient Dynamic Density Estimator for Data Streams

Qahtan, Abdulhakim; Wang, Suojin; Zhang, Xiangliang

doi:10.1109/tkde.2016.2626441

Cited by 34 publications

(30 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each curve needs to be normalized first and then summed up by the kernel estimator function to estimate the density of sea clutters. The expression of KDE can be written as follows [12] f…”

Section: Problem Formulationmentioning

confidence: 99%

Sea Clutter Distribution Modeling: A Kernel Density Estimation Approach

Zhou

Jiang

2018

2018 10th International Conference on Wireless Communications and Signal Processing (WCSP)

View full text Add to dashboard Cite

An accurate sea clutter distribution is crucial for decision region determination when detecting sea-surface floating targets. However, traditional parametric models possibly have a considerable gap to the realistic distribution of sea clutters due to the volatile sea states. In this paper, we develop a kernel density estimation based framework to model the sea clutter distributions without requiring any prior knowledge. In this framework, we jointly consider two embedded fundamental problems, the selection of a proper kernel density function and the determination of its corresponding optimal bandwidth. Regarding these two problems, we adopt the Gaussian, Gamma, and Weibull distributions as the kernel functions, and derive the closed-form optimal bandwidth equations for them. To deal with the highly complicated equations for the three kernels, we further design a fast iterative bandwidth selection algorithm to solve them. Experimental results show that, compared with existing methods, our proposed approach can significantly decrease the error incurred by sea clutter modeling (about two orders of magnitude reduction) and improve the target detection probability (up to 36% in low false alarm rate cases).

show abstract

“…Each curve needs to be normalized first and then summed up by the kernel estimator function to estimate the density of sea clutters. The expression of KDE can be written as follows [12] f…”

Section: Problem Formulationmentioning

confidence: 99%

Sea Clutter Distribution Modeling: A Kernel Density Estimation Approach

Zhou

Jiang

2018

2018 10th International Conference on Wireless Communications and Signal Processing (WCSP)

View full text Add to dashboard Cite

show abstract

“…The interpolation error will increase in the case of multi-dimensional data. In [18], we derived the interpolation error in 2-dimension asf (a a a) −f (a a a) =…”

Section: Density Estimation By Interpolationmentioning

confidence: 99%

“…This section evaluates the most popular density estimators, including 1) the traditional KDE [20] defined in Eq. (1); 2) the FFT-KDE [20,19], which deploys FFT to convolve a very fine histogram of the data with a kernel function to produce a continuous density function; 3) the Cluster Kernels (CK) [10], which maintains a specific number of kernels by merging similar kernels; 4) SOMKE [12], which employs SOM to cluster the data into a specific number of clusters and uses the centroids of the clusters as the set of kernels; and 5) KDE-Track [18] presented in Section 3.2.…”

Section: Density Estimation Performance Evaluationmentioning

confidence: 99%

“…In this chapter, we studied the problem of estimating the dynamic density that comes with data streams. We presented KDE-Track [18] that can timely track the evolving distribution and accurately estimate the probability density function of evolving data streams. KDE-Track models the data distribution as a set of resampling points with their estimated PDF values, which are incrementally updated upon the arrival of new data samples from the stream.…”

Section: Summary and Future Workmentioning

confidence: 99%

See 1 more Smart Citation

Efficient Estimation of Dynamic Density Functions with Applications in Data Streams

Qahtan¹,

Wang²,

Zhang

2018

Studies in Big Data

Self Cite

View full text Add to dashboard Cite

Recently, many applications such as network monitoring, traffic management and environmental studies generate huge amount of data that cannot fit in the computer memory. Data of such applications arrive continuously in the form of streams. The main challenges for mining data streams are the high speed and the large volume of the arriving data. A typical solution to tackle the problems of mining data streams is to learn a model that fits in the computer memory. However, the underlying distributions of the streaming data change over time in unpredicted scenarios. In this sense, the learned models should be updated continuously and rely more on the most recent data in the streams. In this chapter, we present an online density estimator that builds a model called KDE-Track for characterizing the dynamic density of the data streams. KDE-Track summarizes the distribution of a data stream by estimating the Probability Density Function (PDF) of the stream at a set of resampling points. KDE-Track is shown to be more accurate (as reflected by smaller error values) and more computationally efficient (as reflected by shorter running time) when compared with existing density estimation techniques. We demonstrate the usefulness of KDE-Track in visualizing the dynamic density of data streams and change detection.

show abstract

“…In our paper [3], we presented a method, called KDE-Track, to model the data distribution as a set of resampling points with their estimated PDF. To guarantee the estimation accuracy and to lighten the load on the model, an adaptive resampling strategy is employed to control the number of resampling points, i.e., more points are resampled in the areas where the PDF has a larger curvature, while less number of points are resampled in the areas where the function is approximately linear.…”

Section: Introductionmentioning

confidence: 99%

KDE-Track: An Efficient Dynamic Density Estimator for Data Streams (Extended Abstract)

Qahtan¹,

Wang

Zhang

2018

2018 IEEE 34th International Conference on Data Engineering (ICDE)

Self Cite

View full text Add to dashboard Cite

Recent developments in sensors, global positioning system devices and smart phones have increased the availability of spatiotemporal data streams. Developing models for mining such streams is challenged by the huge amount of data that cannot be stored in the memory, the high arrival speed and the dynamic changes in the data distribution. Density estimation is an important technique in stream mining for a wide variety of applications. In this paper, we present a method called KDE-Track to estimate the density of spatiotemporal data streams. KDE-Track can efficiently estimate the density function with linear time complexity using interpolation on a kernel model, which is incrementally updated upon the arrival of new samples from the stream.

show abstract

KDE-Track: An Efficient Dynamic Density Estimator for Data Streams

Cited by 34 publications

References 41 publications

Sea Clutter Distribution Modeling: A Kernel Density Estimation Approach

Sea Clutter Distribution Modeling: A Kernel Density Estimation Approach

Efficient Estimation of Dynamic Density Functions with Applications in Data Streams

KDE-Track: An Efficient Dynamic Density Estimator for Data Streams (Extended Abstract)

Contact Info

Product

Resources

About