Denoised Internal Models: A Brain-inspired Autoencoder Against Adversarial Attacks

Liu, Kaiyuan; Li, Xingyu; Zhou, Yi; Guan, Ji‐Song; Lai, Yurui; Zhang, Ge; Su, Hang; Wang, Jiachen; Guo, Chunxu

doi:10.1007/s11633-022-1375-7

Cited by 4 publications

(1 citation statement)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The primary function of the hippocampus is that it enables long-term memory of the spatial and sequential order from the human experience (Bird and Burgess, 2008;Davachi and DuBrow, 2015). This property makes the human intelligence exhibits robust and performs better than machine intelligence (Goodfellow et al, 2014;Zhou and Firestone, 2019;Liu et al, 2021). Meanwhile, ANNs trained with backpropagation tend to forget what it learned when it learns new information, that is catastrophic forgetting (McCloskey and Cohen, 1989;French, 1999;Goodfellow et al, 2013).…”

Section: Introductionmentioning

confidence: 99%

Brain-inspired Predictive Coding Improves the Performance of Machine Challenging Tasks

Lee

Jo²,

Lee³

et al. 2022

Front. Comput. Neurosci.

View full text Add to dashboard Cite

Backpropagation has been regarded as the most favorable algorithm for training artificial neural networks. However, it has been criticized for its biological implausibility because its learning mechanism contradicts the human brain. Although backpropagation has achieved super-human performance in various machine learning applications, it often shows limited performance in specific tasks. We collectively referred to such tasks as machine-challenging tasks (MCTs) and aimed to investigate methods to enhance machine learning for MCTs. Specifically, we start with a natural question: Can a learning mechanism that mimics the human brain lead to the improvement of MCT performances? We hypothesized that a learning mechanism replicating the human brain is effective for tasks where machine intelligence is difficult. Multiple experiments corresponding to specific types of MCTs where machine intelligence has room to improve performance were performed using predictive coding, a more biologically plausible learning algorithm than backpropagation. This study regarded incremental learning, long-tailed, and few-shot recognition as representative MCTs. With extensive experiments, we examined the effectiveness of predictive coding that robustly outperformed backpropagation-trained networks for the MCTs. We demonstrated that predictive coding-based incremental learning alleviates the effect of catastrophic forgetting. Next, predictive coding-based learning mitigates the classification bias in long-tailed recognition. Finally, we verified that the network trained with predictive coding could correctly predict corresponding targets with few samples. We analyzed the experimental result by drawing analogies between the properties of predictive coding networks and those of the human brain and discussing the potential of predictive coding networks in general machine learning.

show abstract