Is a Complex-Valued Stepsize Advantageous in Complex-Valued Gradient Learning Algorithms?

Zhang, Huisheng; Mandic, Danilo P.

doi:10.1109/tnnls.2015.2494361

Cited by 43 publications

(18 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example [34], inspired by the theory of widely linear adaptive filters, augments the input to the CVNN with its complex conjugate x * . Additional improvements can be obtained by replacing the real-valued µ with a complex-valued learning rate [35], which can speed up convergence in some scenarios.…”

Section: B Complex-valued Neural Networkmentioning

confidence: 99%

Complex-Valued Neural Networks With Nonparametric Activation Functions

Scardapane

Vaerenbergh

Hussain

et al. 2020

IEEE Trans. Emerg. Top. Comput. Intell.

View full text Add to dashboard Cite

Complex-valued neural networks (CVNNs) are a powerful modeling tool for domains where data can be naturally interpreted in terms of complex numbers. However, several analytical properties of the complex domain (e.g., holomorphicity) make the design of CVNNs a more challenging task than their real counterpart. In this paper, we consider the problem of flexible activation functions (AFs) in the complex domain, i.e., AFs endowed with sufficient degrees of freedom to adapt their shape given the training data. While this problem has received considerable attention in the real case, a very limited literature exists for CVNNs, where most activation functions are generally developed in a split fashion (i.e., by considering the real and imaginary parts of the activation separately) or with simple phase-amplitude techniques. Leveraging over the recently proposed kernel activation functions (KAFs), and related advances in the design of complex-valued kernels, we propose the first fully complex, non-parametric activation function for CVNNs, which is based on a kernel expansion with a fixed dictionary that can be implemented efficiently on vectorized hardware. Several experiments on common use cases, including prediction and channel equalization, validate our proposal when compared to real-valued neural networks and CVNNs with fixed activation functions.

show abstract

Section: B Complex-valued Neural Networkmentioning

confidence: 99%

Complex-Valued Neural Networks With Nonparametric Activation Functions

Scardapane

Vaerenbergh

Hussain

et al. 2020

IEEE Trans. Emerg. Top. Comput. Intell.

View full text Add to dashboard Cite

show abstract

“…where det(•) denotes the matrix determinant operator, the augmented covariance matrices C x a x a and C xx are defined in Section III.A, and the index ρ is normalized within [0, 1] with the value 0 indicating perfect circularity. According to (53), it can be calculated that the noncircularity of the two benchmark problems are 0.8936 and 0.4253 respectively, which indicates that the noncircular chaotic Ikeda map signal is of great noncircularity, whereas the other one is of less noncircularity. The CFNNSLs used in simulations are single output networks, and input weights and hidden biases are randomly generated by drawing the real parts and imaginary parts from a uniform distribution U (−0.1, 0.1).…”

Section: Simulation Resultsmentioning

confidence: 99%

“…where λ > 0 is the regularization parameter to balance the tradeoff between the training error and the output weight norm. According to Wirtinger calculus theory [41], [51]- [53], the solution of this optimization problem can be obtained by solving…”

Section: A Regularized Os-celmmentioning

confidence: 99%

Online Sequential Complex-Valued ELM for Noncircular Signals: Augmented Structures and Learning Algorithms

Wang

Zhu

et al. 2021

IEEE Access

Self Cite

View full text Add to dashboard Cite

Online sequential extreme learning machine (OS-ELM) has become a popular online learning strategy for single-hidden layer feedforward neural networks, and complex-valued signals are ubiquitous in real applications. In order to cater for complex-valued signals, especially for noncircular signals, in this paper we extend OS-ELM to complex domain, and propose two augmented online sequential complex ELM models by incorporating the conjugates of the network input and the hidden layer respectively. In this way, the proposed models are equipped with the capability to capture the complete second-order statistics of noncircular signals, which results in the enhanced generalization ability. The corresponding regularized models are derived to avoid the possible overfitting problems. By exploiting the algebra structure resulted from the augmented architecture, several approaches to reducing the computational complexity are also proposed. Simulation results validate the efficiency of the proposed algorithms. INDEX TERMSComplex extreme learning machine (CELM), Online sequential learning algorithms, Noncircular signals, Augmented complex statistics, Computational complexity

show abstract

“…Whereas BFGS algorithm also requires a number of storage units for each update encountering large-scale optimization problems, which weakens the memory efficiency of BFGS. Limited-memory BFGS (LBFGS) was further introduced, and also extended to the complex-valued domain in [29]. Compared with BFGS, LBFGS only needs to store the information of the latest m iterations to approximate the inverse of Hessian matrix, releasing the pressure of storage and showing outstanding convergence performance, both in real and complex domain [30].…”

Section: The Features Of Cvdnnmentioning

confidence: 99%

Optical Phase Conjugation With Complex-Valued Deep Neural Network for WDM 64-QAM Coherent Optical Systems

et al. 2021

View full text Add to dashboard Cite

We experimentally demonstrated a photoelectric nonlinear compensation scheme of optical phase conjugation (OPC) with complex-valued deep neural network (CVDNN) to mitigate fiber nonlinearity in wavelength division multiplexing (WDM) 64-QAM coherent optical transmission system. The factors to affect the performance of OPC and CVDNN are comprehensively considered. OPC in WDM system is experimentally optimized to alleviate the deployment requirements of strict symmetrical distributed power and chromatic dispersion. The performance penalty caused by the simplification of the OPC is further compensated by the CVDNN. The selections of the input neurons' number and the optimization algorithm are also considered to design a simple two-hidden-layerstructure CVDNN. The proposed method is experimentally verified and evaluated in a 12.5-GBd 4-channel WDM 64-QAM 160-km standard single-mode fiber (SSMF) transmission system with channel spacing of 50-GHz. The experimental results show that the proposed nonlinear equalizer based on the OPC with CDVNN has a strong robustness to the input signal power and wavelength, which can not only improve the Q factor of the signal by 1.5-dB, but also greatly increase the total launched signal power.

show abstract

Is a Complex-Valued Stepsize Advantageous in Complex-Valued Gradient Learning Algorithms?

Cited by 43 publications

References 32 publications

Complex-Valued Neural Networks With Nonparametric Activation Functions

Complex-Valued Neural Networks With Nonparametric Activation Functions

Online Sequential Complex-Valued ELM for Noncircular Signals: Augmented Structures and Learning Algorithms

Optical Phase Conjugation With Complex-Valued Deep Neural Network for WDM 64-QAM Coherent Optical Systems

Contact Info

Product

Resources

About