Few-Shot Learning with Complex-Valued Neural Networks and Dependable Learning

Wang, Runqi; Liu, Zhen; Zhang, Baochang; Guo, Guodong; Doermann, David

doi:10.1007/s11263-022-01700-x

Cited by 4 publications

(2 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A summary of the above areas of applications is in Table 5. [136], SNN [137], CVNN [138], AE [139] These optimizers are fast and can easily be customized. They still meet in many modern neural networks.…”

Section: Application Of Optimization Methods In Modern Neural Networkmentioning

confidence: 99%

Survey of Optimization Algorithms in Modern Neural Networks

2023

View full text Add to dashboard Cite

The main goal of machine learning is the creation of self-learning algorithms in many areas of human activity. It allows a replacement of a person with artificial intelligence in seeking to expand production. The theory of artificial neural networks, which have already replaced humans in many problems, remains the most well-utilized branch of machine learning. Thus, one must select appropriate neural network architectures, data processing, and advanced applied mathematics tools. A common challenge for these networks is achieving the highest accuracy in a short time. This problem is solved by modifying networks and improving data pre-processing, where accuracy increases along with training time. Bt using optimization methods, one can improve the accuracy without increasing the time. In this review, we consider all existing optimization algorithms that meet in neural networks. We present modifications of optimization algorithms of the first, second, and information-geometric order, which are related to information geometry for Fisher–Rao and Bregman metrics. These optimizers have significantly influenced the development of neural networks through geometric and probabilistic tools. We present applications of all the given optimization algorithms, considering the types of neural networks. After that, we show ways to develop optimization algorithms in further research using modern neural networks. Fractional order, bilevel, and gradient-free optimizers can replace classical gradient-based optimizers. Such approaches are induced in graph, spiking, complex-valued, quantum, and wavelet neural networks. Besides pattern recognition, time series prediction, and object detection, there are many other applications in machine learning: quantum computations, partial differential, and integrodifferential equations, and stochastic processes.

show abstract

Section: Application Of Optimization Methods In Modern Neural Networkmentioning

confidence: 99%

Survey of Optimization Algorithms in Modern Neural Networks

2023

View full text Add to dashboard Cite

show abstract

“…Optimizer Application SGD-type CNN [131], RNN [132], GNN [133], SGD PINN [134], SNN [135], CVNN [136], AE [137] AdaGrad CNN [138] AdaDelta CNN [139], RNN [139], SNN [140] RMSProp CNN [141], RNN [142], SNN [ CNN [144], RNN [145] Adam-type CNN [146], RNN [147], SNN [148], Adam PINN [150], GNN [151], CVNN [ Information geometry CNN [168], RNN [109], NGD GNN [169], PINN [170], QNN [170] MD CNN, RNN [172] Note, that CNN, RNN, GNN, PINN, SNN, CVNN, QNN are convolutional, recurrent, graph, physics-informed, spiking, complex-valued and quantum neural networks, respectively. These neural networks proved their ability to solve vast majority of problems, related to recognition, prediction, generation, processing, detecting and so on.…”

Section: Type Of Optimization Algorithmmentioning

confidence: 99%

Survey of Optimization Algorithms in Modern Neural Networks

Abdulkadirov¹,

Lyakhov²,

Nagornov³

2023

Preprint

View full text Add to dashboard Cite

Creating self-learning algorithms, developing deep neural networks and improving other methods that "learn" for various areas of human activity is the main goal of the theory of machine learning. It helps to replace the human with a machine, aiming to increase the quality of production. The theory of artificial neural networks, which already have replaced the humans in problems of detection of moving objects, recognition of images or sounds, time series prediction, big data analysis and numerical methods remains the most dispersed branch of the theory of machine learning. Certainly, for each area of human activity it is necessary to select appropriate neural network architectures, methods of data processing and some novel tools from applied mathematics. But the universal problem for all these neural networks with specific data is the achieving the highest accuracy in short time. Such problem can be resolved by increasing sizes of architectures and improving data preprocessing, where the accuracy rises with the training time. But there is a possibility to increase the accuracy without time growing, applying optimization methods. In this survey we demonstrate existing optimization algorithms of all types, which can be used in neural networks. There are presented modifications of basic optimization algorithms, such as stochastic gradient descent, adaptive moment estimation, Newton and quasi-Newton optimization methods. But the most recent optimization algorithms are related to information geometry, for Fisher-Rao and Bregman metrics. This approach in optimization extended the theory of classic neural networks to quantum and complex-valued neural networks, due to geometric and probabilistic tools. There are provided applications of all introduced optimization algorithms, what delighted many kinds of neural networks, which can be improved by including any advanced approaches in minimization of the loss function. Afterwards, we demonstrated ways of developing optimization algorithms in further researches, engaging neural networks with progressive architectures. Classical gradient based optimizers can be replaced by fractional order, bilevel and, even, gradient free optimization methods. There is a possibility to add such analogues in graph, spiking, complex-valued, quantum and wavelet neural networks. Besides the usual problems of image recognition, time series prediction, object detection, there are many are other tasks for modern theory of machine learning, such as solving problem of quantum computations, partial differential and integro-differential equations, stochastic processes and Brownian motion, making decisions and computer algebra.

show abstract

Contrastive prototype network with prototype augmentation for few-shot classification

Jiang,

Fan,

et al. 2025

Information Sciences

View full text Add to dashboard Cite

Few-Shot Learning with Complex-Valued Neural Networks and Dependable Learning

Cited by 4 publications

References 47 publications

Survey of Optimization Algorithms in Modern Neural Networks

Survey of Optimization Algorithms in Modern Neural Networks

Survey of Optimization Algorithms in Modern Neural Networks

Contrastive prototype network with prototype augmentation for few-shot classification

Contact Info

Product

Resources

About