Convergence and Alignment of Gradient Descent with Random Backpropagation Weights

Abstract

Stochastic gradient descent with backpropagation is the workhorse of artificial neural networks. It has long been recognized that backpropagation fails to be a biologically plausible algorithm. Fundamentally, it is a non-local procedure-updating one neuron's synaptic weights requires knowledge of synaptic weights or receptive fields of downstream neurons. This limits the use of artificial neural networks as a tool for understanding the biological principles of information processing in the brain. Lillicrap et al. (2016) propose a more biologically plausible "feedback alignment" algorithm that uses random and fixed backpropagation weights, and show promising simulations and analysis. In this paper we study the mathematical properties of the feedback alignment procedure by analyzing convergence and alignment for two-layer networks under squared error loss. In the overparameterized setting, we prove that the error converges to zero exponentially fast, and also that regularization is necessary in order for the parameters to become aligned with the random backpropagation weights. Simulations are given that are consistent with this analysis and suggest further generalizations. These results contribute to our understanding of how biologically plausible algorithms might carry out weight learning in a manner different from Hebbian learning, with performance that is comparable with the full non-local backpropagation algorithm.

How and When Random Feedback Works: A Case Study of Low-Rank Matrix Factorization

“…A natural next step is to extend our understanding of alignment dynamics to the problem of learning neural networks. Song et al (2021) show that alignment may not happen in highly overparameterized neural networks. But it is a robust phenomenon in the parameter regimes typically encountered in practice, and therefore important to understand.…”

“…In recent work, (Song et al, 2021) study feedback alignment for highly overparameterized onehidden layer neural networks where the width of the hidden layer is much larger than the size of training set. This work builds on past work on Neural Tangent Kernels (Jacot et al, 2018), and shows that feedback alignment converges to a solution with zero training error.…”

Cascaded Gaps: Towards Gap-Dependent Regret for Risk-Sensitive Reinforcement Learning

“…Understanding and designing efficient algorithms for risk-sensitive RL in other settings, such as shortest path problems (Min et al, 2021b), off-policy evaluation (Min et al, 2021c) and offline learning (Chen et al, 2021b), may also be of great interest. Moreover, as risk-sensitive RL is closely related to human learning and behaviors, it would be intriguing to study how it synthesizes with relevant areas such as meta learning and bio-inspired learning (Song et al, 2021;Xu et al, 2021). Last but not least, exploring how risk sensitivity could be used to augment unsupervised learning algorithms (Fei & Chen, 2018a,b, 2020Ling et al, 2019) would be an important future topic as well.…”

Exponential Bellman Equation and Improved Regret Bounds for Risk-Sensitive Reinforcement Learning

“…≤ (q k,+ h,2 − q k h,3 )(s, a) = α 0 t e β(H−h+1) − e of meta learning [44], biologically inspired deep learning [42] and deep reinforcement learning [29]. It would be an exciting research direction to establish connections between these related areas through rigorous and theoretical analysis of deep learning [9,11].…”