In-sensor neural network for high energy efficiency analog-to-information conversion

In-sensor and near-sensor computing architectures enable multiply-accumulate operations to be carried out directly at the point of sensing. In-sensor architectures offer dramatic power and speed improvements over traditional von Neumann architectures by eliminating multiple analog-todigital conversions, data storage, and data movement operations. Current in-sensor processing approaches rely on tunable sensors or additional weighting elements to perform linear functions such as multiply-accumulate operations as the sensor acquires data. We implement in-sensor computing with an oscillatory retinal neuron device that converts incident optical signals into voltage oscillations. We introduce a computing scheme based on the frequency shift of coupled oscillators that enables parallel, frequency multiplexed, non-linear operations on the inputs. Simulation elucidates how this computing occurs. We experimentally implement a 3×3 focal plane array of coupled neurons and show that functions approximating edge detection, thresholding, and segmentation occur in parallel. An example of inference on handwritten digits from the MNIST database is also experimentally demonstrated with a 3×3 array of coupled neurons feeding into a

show abstract

Analog spatiotemporal feature extraction for cognitive radio-frequency sensing with integrated photonics

Xu,

Liu,

et al. 2024

Light Sci Appl

View full text Add to dashboard Cite

Analog feature extraction (AFE) is an appealing strategy for low-latency and efficient cognitive sensing systems since key features are much sparser than the Nyquist-sampled data. However, applying AFE to broadband radio-frequency (RF) scenarios is challenging due to the bandwidth and programmability bottlenecks of analog electronic circuitry. Here, we introduce a photonics-based scheme that extracts spatiotemporal features from broadband RF signals in the analog domain. The feature extractor structure inspired by convolutional neural networks is implemented on integrated photonic circuits to process RF signals from multiple antennas, extracting valid features from both temporal and spatial dimensions. Because of the tunability of the photonic devices, the photonic spatiotemporal feature extractor is trainable, which enhances the validity of the extracted features. Moreover, a digital-analog-hybrid transfer learning method is proposed for the effective and low-cost training of the photonic feature extractor. To validate our scheme, we demonstrate a radar target recognition task with a 4-GHz instantaneous bandwidth. Experimental results indicate that the photonic analog feature extractor tackles broadband RF signals and reduces the sampling rate of analog-to-digital converters to 1/4 of the Nyquist sampling while maintaining a high target recognition accuracy of 97.5%. Our scheme offers a promising path for exploiting the AFE strategy in the realm of cognitive RF sensing, with the potential to contribute to the efficient signal processing involved in applications such as autonomous driving, robotics, and smart factories.

show abstract

In-sensor neural network for high energy efficiency analog-to-information conversion

Cited by 9 publications

References 38 publications

Electrochemical reservoir computing based on surface-functionalized carbon nanotubes

Electrochemical reservoir computing based on surface-functionalized carbon nanotubes

Ultralow Power In-Sensor Neuronal Computing with Oscillatory Retinal Neurons for Frequency-Multiplexed, Parallel Machine Vision

Analog spatiotemporal feature extraction for cognitive radio-frequency sensing with integrated photonics

Contact Info

Product

Resources

About