Embodiment of Learning in Electro-Optical Signal Processors

Hermans, Michiel; Antonik, Piotr; Haelterman, Marc; Massar, Serge

doi:10.1103/physrevlett.117.128301

Cited by 33 publications

(20 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Equation (15) shows that for C > 0 the Hamming distance between readout weights of two systems will always tend towards complete decorrelation as H(k)| k→∞ N/2. Even for 100% identical networks one will therefore never obtain similar readout configurations [29].…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Boolean learning under noise-perturbations in hardware neural networks

et al. 2020

View full text Add to dashboard Cite

AbstractA high efficiency hardware integration of neural networks benefits from realizing nonlinearity, network connectivity and learning fully in a physical substrate. Multiple systems have recently implemented some or all of these operations, yet the focus was placed on addressing technological challenges. Fundamental questions regarding learning in hardware neural networks remain largely unexplored. Noise in particular is unavoidable in such architectures, and here we experimentally and theoretically investigate its interaction with a learning algorithm using an opto-electronic recurrent neural network. We find that noise strongly modifies the system’s path during convergence, and surprisingly fully decorrelates the final readout weight matrices. This highlights the importance of understanding architecture, noise and learning algorithm as interacting players, and therefore identifies the need for mathematical tools for noisy, analogue system optimization.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Noise is an inseparable companion of analogue hardware [14], yet the fundamental aspects of optimizing a noisy NN [15][16][17] have so far hardly been exploredneither in experiments [18,19] nor in theory [14]. Here, we investigate the interactions between noise, learning rules and the topology of an error landscape for the first time.…”

Section: Introductionmentioning

confidence: 99%

Boolean learning under noise-perturbations in hardware neural networks

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Information is retained within this system by maintaining transience in the system through input time multiplexing [22]. Delay-based RC has been demonstrated using Boolean nodes [24], photonics [25][26][27][28], spin-torque oscillators [29], coupled oscillators [30], magnet arrays [31], and recently, MEMS devices [32,33]. Among these physical devices, photonics are the most popular due to their extremely high processing speeds.…”

Section: Literature Reviewmentioning

confidence: 99%

“…is some update function for the dynamical system state, t is time, t * is some past time value(s) and I(t) is an external input term. High coupling between the virtual nodes is achieved by retaining the dynamical system transience [28] through input time multiplexing [22]. Time multiplexing is usually performed using a sequence of analog-to-digital conversion, sampling-and-holding (for the duration of τ), and masking.…”

Section: Reservoir Computingmentioning

confidence: 99%

Colocalized Sensing and Intelligent Computing in Micro-Sensors

Hasan

Al-Ramini

Abdel‐Rahman

et al. 2020

Sensors

View full text Add to dashboard Cite

This work presents an approach to delay-based reservoir computing (RC) at the sensor level without input modulation. It employs a time-multiplexed bias to maintain transience while utilizing either an electrical signal or an environmental signal (such as acceleration) as an unmodulated input signal. The proposed approach enables RC carried out by sufficiently nonlinear sensory elements, as we demonstrate using a single electrostatically actuated microelectromechanical system (MEMS) device. The MEMS sensor can perform colocalized sensing and computing with fewer electronics than traditional RC elements at the RC input (such as analog-to-digital and digital-to-analog converters). The performance of the MEMS RC is evaluated experimentally using a simple classification task, in which the MEMS device differentiates between the profiles of two signal waveforms. The signal waveforms are chosen to be either electrical waveforms or acceleration waveforms. The classification accuracy of the presented MEMS RC scheme is found to be over 99%. Furthermore, the scheme is found to enable flexible virtual node probing rates, allowing for up to 4× slower probing rates, which relaxes the requirements on the system for reservoir signal sampling. Finally, our experiments show a noise-resistance capability for our MEMS RC scheme.

show abstract

“…A first implementation of back propagation training in an optoelectronic system was demonstrated by Hermans et al 94 . More recently, an experimental photonic demonstration 90 was based on an optoelectronic system consisting of a superluminescent diode, two coupled Mach-Zehnder modulators, a long spool of fiber as an optical delay line, interfaced with an FPGA to generate and record signals, and a computer controlling the experiment (Fig. 10).…”

Section: H Back Propagation Through Timementioning

confidence: 99%

Tutorial: Photonic neural networks in delay systems

Brunner

Penkovsky

Márquez

et al. 2018

Journal of Applied Physics

136

View full text Add to dashboard Cite

Photonic delay systems have revolutionized the hardware implementation of Recurrent Neural Networks and Reservoir Computing in particular. The fundamental principles of Reservoir Computing strongly benefit a realization in such complex analog systems. Especially delay systems, potentially providing large numbers of degrees of freedom even in simple architectures, can efficiently be exploited for information processing. The numerous demonstrations of their performance led to a revival of photonic Artificial Neural Network. Today, an astonishing variety of physical substrates, implementation techniques as well as network architectures based on this approach have been successfully employed. Important fundamental aspects of analog hardware Artificial Neural Networks have been investigated, and multiple high-performance applications have been demonstrated. Here, we introduce and explain the most relevant aspects of Artificial Neural Networks and delay systems, the seminal experimental demonstrations of Reservoir Computing in photonic delay systems, plus the most recent and advanced realizations.

show abstract

Embodiment of Learning in Electro-Optical Signal Processors

Cited by 33 publications

References 25 publications

Boolean learning under noise-perturbations in hardware neural networks

Boolean learning under noise-perturbations in hardware neural networks

Colocalized Sensing and Intelligent Computing in Micro-Sensors

Tutorial: Photonic neural networks in delay systems

Contact Info

Product

Resources

About