The Online Random Fourier Features Conjugate Gradient Algorithm

Xiong, Kui; Wang, Shiyuan

doi:10.1109/lsp.2019.2907480

Cited by 29 publications

(12 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Random Fourier features have been used successfully to increase the scalability of kernel-based methods such as support vector machines (SVMs) [34]. Introducing randomization to the learning process has also been studied in machine learning literature [35].…”

Section: B Prior Art and Comparisonsmentioning

confidence: 99%

Modeling of Spatio-Temporal Hawkes Processes With Randomized Kernels

İlhan

Kozat

2020

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

We investigate spatio-temporal event analysis using point processes. Inferring the dynamics of event sequences spatiotemporally has many practical applications including crime prediction, social media analysis, and traffic forecasting. In particular, we focus on spatio-temporal Hawkes processes that are commonly used due to their capability to capture excitations between event occurrences. We introduce a novel inference framework based on randomized transformations and gradient descent to learn the process. We replace the spatial kernel calculations by randomized Fourier feature-based transformations. The introduced randomization by this representation provides flexibility while modeling the spatial excitation between events. Moreover, the system described by the process is expressed within closed-form in terms of scalable matrix operations. During the optimization, we use maximum likelihood estimation approach and gradient descent while properly handling positivity and orthonormality constraints. The experiment results show the improvements achieved by the introduced method in terms of fitting capability in synthetic and real-life datasets with respect to the conventional inference methods in the spatio-temporal Hawkes process literature. We also analyze the triggering interactions between event types and how their dynamics change in space and time through the interpretation of learned parameters.

show abstract

Section: B Prior Art and Comparisonsmentioning

confidence: 99%

Modeling of Spatio-Temporal Hawkes Processes With Randomized Kernels

İlhan

Kozat

2020

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

show abstract

“…where positive constant 0 < λ < 1 is usually closest to one, x k ∈ R n is the input data, and d k is the desired output at iteration k. Define a residual vector of normal equations as s k = b k − R k ω k at iteration k. To improve clarity, the online CG algorithm to estimate the weight vector is summarized as follows [15]. Given the initial conditions ω 0 = 0 and…”

Section: Online Conjugate Gradient Algorithmmentioning

confidence: 99%

“…To demonstrate the superior performance of the proposed RFFCCG algorithm in this section, simulations were performed on the Mackey-Glass chaotic time series prediction and nonlinear system identification, respectively. Due to the modest complexity and excellent performance, representative algorithms (i.e., random Fourier features kernel least mean square (RFFKLMS) algorithm [13], quantized kernel recursive least squares (QKRLS) algorithm [32], random Fourier features maximum correntropy (RFFMC) algorithm [14], kernel recursive maximum correntropy algorithm with novelty criterion (KRMC-NC) [31], and random Fourier features conjugate gradient (RFFCG) algorithm [15]) were selected to compare the performance of RFFCCG. Among these algorithms, RFFMC and KRMC-NC are typical robust algorithms, while RFFKLMS, QKRLS, and RFFCG with no robustness are also used for the filtering performance reference.…”

Section: Simulationmentioning

confidence: 99%

“…The RFM alleviates the computational and storage burdens of KAFs, and ensures a satisfactory performance under non-stationary conditions. The examples for developing KAFs with RFM are the random Fourier features kernel least mean square (RFFKLMS) algorithm [13], random Fourier features maximum correntropy (RFFMC) algorithm [14], and random Fourier features conjugate gradient (RFFCG) algorithm [15].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The Cauchy Conjugate Gradient Algorithm with Random Fourier Features

Huang

Wang²,

Xiong³

2019

Symmetry

Self Cite

View full text Add to dashboard Cite

Random Fourier mapping (RFM) in kernel adaptive filters (KAFs) provides an efficient method to curb the linear growth of the dictionary by projecting the original input data into a finite-dimensional space. The commonly used measure in RFM-based KAFs is the minimum mean square error (MMSE), which causes performance deterioration in the presence of non-Gaussian noises. To address this issue, the minimum Cauchy loss (MCL) criterion has been successfully applied for combating non-Gaussian noises in KAFs. However, these KAFs using the well-known stochastic gradient descent (SGD) optimization method may suffer from slow convergence rate and low filtering accuracy. To this end, we propose a novel robust random Fourier features Cauchy conjugate gradient (RFFCCG) algorithm using the conjugate gradient (CG) optimization method in this paper. The proposed RFFCCG algorithm with low complexity can achieve better filtering performance than the KAFs with sparsification, such as the kernel recursive maximum correntropy algorithm with novelty criterion (KRMC-NC), in stationary and non-stationary environments. Monte Carlo simulations conducted in the time-series prediction and nonlinear system identification confirm the superiorities of the proposed algorithm.

show abstract

“…The vector quantization method is the improvement of sparsification method, which utilizes the information of redundant data discarded by the sparsification method at the cost of increasing computation cost [36], [37]. Unlike sparsification and vector quantization methods, the RFF method is data-independent and provides a fixed-dimensional network structure [38]- [40]. The main drawback of RFF method is that its approximation accuracy depends on the dimensionality which needs to be determined for different applications in advance.…”

Section: Introductionmentioning

confidence: 99%

Multikernel Adaptive Filters Under the Minimum Cauchy Kernel Loss Criterion

2019

Self Cite

View full text Add to dashboard Cite

The Cauchy loss has been successfully applied in robust learning algorithms in the presence of large outliers, but it may suffer from performance degradation in complex nonlinear tasks. To address this issue, by transforming the original data into the reproducing kernel Hilbert spaces (RKHS) with the kernel trick, a novel Cauchy kernel loss is developed in such a kernel space. Based on the minimum Cauchy kernel loss criterion, the multikernel minimum Cauchy kernel loss (MKMCKL) algorithm is proposed by mapping the input data into the multiple RKHS. The proposed MKMCKL algorithm can provide the performance improvement of the kernel adaptive filter (KAF) based on a single kernel, and also improve the stability of the multikernel adaptive filter based on the quadratic loss in impulsive noises, efficiently. To further curb the growth of network of MKMCKL, a novel sparsification method is presented to prune redundant data, thus reducing its computational and storage burdens. Simulations on different nonlinear applications illustrate the performance superiorities of the proposed algorithms in impulsive noises.

show abstract

The Online Random Fourier Features Conjugate Gradient Algorithm

Cited by 29 publications

References 21 publications

Modeling of Spatio-Temporal Hawkes Processes With Randomized Kernels

Modeling of Spatio-Temporal Hawkes Processes With Randomized Kernels

The Cauchy Conjugate Gradient Algorithm with Random Fourier Features

Multikernel Adaptive Filters Under the Minimum Cauchy Kernel Loss Criterion

Contact Info

Product

Resources

About