The Quaternion Stochastic Information Gradient Algorithm for Nonlinear Adaptive Systems

Ogunfunmi, Tokunbo; Safarian, Carlo

doi:10.1109/tsp.2019.2944757

Cited by 22 publications

(3 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Cross-entropy is commonly used as part of loss functions in ML, including FL [90], [91]. In many cases, p is treated as the "true" distribution, and q as the model to be optimized [92].…”

Section: Divergencementioning

confidence: 99%

Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications

Chen

et al. 2020

Preprint

View full text Add to dashboard Cite

Distributed machine learning (DML) techniques, such as federated learning, partitioned learning, and distributed reinforcement learning, have been increasingly applied to wireless communications. This is due to improved capabilities of terminal devices, explosively growing data volume, congestion in the radio interfaces, and increasing concern of data privacy. The unique features of wireless systems, such as large scale, geographically dispersed deployment, user mobility, and massive amount of data, give rise to new challenges in the design of DML techniques. There is a clear gap in the existing literature in that the DML techniques are yet to be systematically reviewed for their applicability to wireless systems. This survey bridges the gap by providing a contemporary and comprehensive survey of DML techniques with a focus on wireless networks. Specifically, we review the latest applications of DML in power control, spectrum management, user association, and edge cloud computing. The optimality, scalability, convergence rate, computation cost, and communication overhead of DML are analyzed. We also discuss the potential adversarial attacks faced by DML applications, and describe state-of-the-art countermeasures to preserve privacy and security. Last but not least, we point out a number of key issues yet to be addressed, and collate potentially interesting and challenging topics for future research.

show abstract

“…Cross-entropy is commonly used as part of loss functions in ML, including FL [90], [91]. In many cases, p is treated as the "true" distribution, and q as the model to be optimized [92].…”

Section: Divergencementioning

confidence: 99%

Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications

Chen

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…For example, a kernel-based gradient descent algorithms based on MEE was proposed to find nonlinear structures in the data, and its convergence rate was deduced [22]. A kernel adaptive filter for quaternion data was developed, and a new algorithm based on the SIG approach was applied to this filter [37]. To avoid unstable training or poor performance in deep learning, a strategy of directly estimating the gradients of information measures with respect to model parameters was explored, and a general gradient estimation method for information measures was proposed [52].…”

Section: Introductionmentioning

confidence: 99%

Identification of the ARX Model with Random Impulse Noise Based on Forgetting Factor Multi-error Information Entropy

Jing

2021

Circuits Syst Signal Process

View full text Add to dashboard Cite

Entropy has been widely applied in system identification in the last decade. In this paper, a novel stochastic gradient algorithm based on minimum Shannon entropy is proposed. Though needing less computation than the mean square error algorithm, the traditional stochastic gradient algorithm converges relatively slowly. To make the convergence faster, a multi-error method and a forgetting factor are integrated into the algorithm. The scalar error is replaced by a vector error with stacked errors. Further, a simple step size method is proposed and a forgetting factor is adopted to adjust the step size. The proposed algorithm is utilized to estimate the parameters of an ARX model with random impulse noise. Several numerical solutions and case study indicate that the proposed algorithm can obtain more accurate estimates than the traditional gradient algorithm and has a faster convergence speed.

show abstract

“…The basic approach on which relies the QVAE is the learning in the quaternion domain, which results in significant advantages in the presence of multidimensional input data (mainly 3D and 4D data) showing some inter-channel correlations. This properties have been widely exploited in shallow learning models, such as linear and nonlinear adaptive filters [ 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 ]. Another fundamental property of quaternion-valued learning is the Hamilton product, which has recently favored the proliferation of convolutional neural networks in the quaternion domain [ 35 , 36 , 37 , 38 ].…”

Section: Introductionmentioning

confidence: 99%

An Information-Theoretic Perspective on Proper Quaternion Variational Autoencoders

Grassucci

Comminiello

Uncini

2021

Entropy

View full text Add to dashboard Cite

Variational autoencoders are deep generative models that have recently received a great deal of attention due to their ability to model the latent distribution of any kind of input such as images and audio signals, among others. A novel variational autoncoder in the quaternion domain H, namely the QVAE, has been recently proposed, leveraging the augmented second order statics of H-proper signals. In this paper, we analyze the QVAE under an information-theoretic perspective, studying the ability of the H-proper model to approximate improper distributions as well as the built-in H-proper ones and the loss of entropy due to the improperness of the input signal. We conduct experiments on a substantial set of quaternion signals, for each of which the QVAE shows the ability of modelling the input distribution, while learning the improperness and increasing the entropy of the latent space. The proposed analysis will prove that proper QVAEs can be employed with a good approximation even when the quaternion input data are improper.

show abstract

The Quaternion Stochastic Information Gradient Algorithm for Nonlinear Adaptive Systems

Cited by 22 publications

References 27 publications

Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications

Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications

Identification of the ARX Model with Random Impulse Noise Based on Forgetting Factor Multi-error Information Entropy

An Information-Theoretic Perspective on Proper Quaternion Variational Autoencoders

Contact Info

Product

Resources

About