Backpropagation-free training of deep physical neural networks

Momeni, Ali; Rahmani, Babak; Malléjac, Matthieu; del Hougne, Philipp; Fleury, Romain

doi:10.1126/science.adi8474

Cited by 22 publications

(1 citation statement)

References 70 publications

(83 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also test the negative-log-likelihood (NLL) of the gener- One of the possible extensions of our work is to train the network physically [49,50]. This becomes critical when an accurate digital modeling of the physical system becomes challenging due to its complexity.…”

mentioning

confidence: 99%

Photonic probabilistic machine learning using quantum vacuum noise

Choi,

Salamin,

Roques-Carmes

et al. 2024

Preprint

View full text Add to dashboard Cite

Probabilistic machine learning utilizes controllable sources of randomness to encode uncertainty and enable statistical modeling. Harnessing the pure randomness of quantum vacuum noise, which stems from fluctuating electromagnetic fields, has shown promise for high speed and energy-efficient stochastic photonic elements. Nevertheless, photonic computing hardware which can control these stochastic elements to program probabilistic machine learning algorithms has been limited. Here, we implement a photonic probabilistic computer consisting of a controllable stochastic photonic element – a photonic probabilistic neuron (PPN). Our PPN is implemented in a bistable optical parametric oscillator (OPO) with vacuum-level injected bias fields. We then program a measurement-and-feedback loop for time-multiplexed PPNs with electronic processors (FPGA or GPU) to solve certain probabilistic machine learning tasks. We showcase probabilistic inference and image generation of MNIST-handwritten digits, which are representative examples of discriminative and generative models. In both implementations, quantum vacuum noise is used as a random seed to encode classification uncertainty or probabilistic generation of samples. In addition, we propose a path towards an all-optical probabilistic computing platform, with an estimated sampling rate of ~ 1 Gbps and energy consumption of ~ 5 fJ/MAC. Our work paves the way for scalable, ultrafast, and energy-efficient probabilistic machine learning hardware.

show abstract

mentioning

confidence: 99%

Photonic probabilistic machine learning using quantum vacuum noise

Choi,

Salamin,

Roques-Carmes

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

Mechanical Neural Networks with Explicit and Robust Neurons

Mei,

Zhou,

Chen

2024

Advanced Science

View full text Add to dashboard Cite

Mechanical computing provides an information processing method to realize sensing‐analyzing‐actuation integrated mechanical intelligence and, when combined with neural networks, can be more efficient for data‐rich cognitive tasks. The requirement of solving implicit and usually nonlinear equilibrium equations of motion in training mechanical neural networks makes computation challenging and costly. Here, an explicit mechanical neuron is developed of which the response can be directly determined without the need of solving equilibrium equations. A training method is proposed to ensure the robustness of the neuron, i.e., insensitivity to defects and perturbations. The explicitness and robustness of the neurons facilitate the assembly of various network structures. Two exemplified networks, a robust mechanical convolutional neural network and a mechanical recurrent neural network with long short‐term memory capabilities for associative learning, are experimentally demonstrated. The introduction of the explicit and robust mechanical neuron streamlines the design of mechanical neural networks fulfilling robotic matter with a level of intelligence.

show abstract

Physical neural networks with self-learning capabilities

Yu,

Guo,

Xiao

et al. 2024

Sci. China Phys. Mech. Astron.

View full text Add to dashboard Cite

Physical neural networks are artificial neural networks that mimic synapses and neurons using physical systems or materials. These networks harness the distinctive characteristics of physical systems to carry out computations effectively, potentially surpassing the constraints of conventional digital neural networks. A recent advancement known as "physical self-learning" aims to achieve learning through intrinsic physical processes rather than relying on external computations. This article offers a comprehensive review of the progress made in implementing physical self-learning across various physical systems. Prevailing learning strategies that contribute to the realization of physical self-learning are discussed. Despite challenges in understanding the fundamental mechanism of learning, this work highlights the progress towards constructing intelligent hardware from the ground up, incorporating embedded self-organizing and self-adaptive dynamics in physical systems.

show abstract

Backpropagation-free training of deep physical neural networks

Cited by 22 publications

References 70 publications

Photonic probabilistic machine learning using quantum vacuum noise

Photonic probabilistic machine learning using quantum vacuum noise

Mechanical Neural Networks with Explicit and Robust Neurons

Physical neural networks with self-learning capabilities

Contact Info

Product

Resources

About