AnalogNet: Convolutional Neural Network Inference on Analog Focal Plane Sensor Processors

Wong, Matthew Z.; Guillard, Benoît; Murai, Riku; Saeedi, Sajad

doi:10.48550/arxiv.2006.01765

Cited by 7 publications

(13 citation statements)

References 17 publications

(21 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…sensor which performs learnt pixel exposures [15], to Convolutional Neural Networks [20], [4]. Its distinctive ability to perform computation on the focal-plane reduces power consumption and data transfers, making the device promising for edge computation.…”

Section: Background: Scamp-5 Focal-plane Sensor-processormentioning

confidence: 99%

“…We will look at a simple example of how a convolutional kernel is represented in Cain. Here we use AnalogNet2 [20] [12] which is a CNN designed for SCAMP-5.…”

Section: Definitionsmentioning

confidence: 99%

“…Neither Cain or AUKE perform a compete exhaustive search. The AnalogNet2 filter is the kernels used in AnalogNet2 [20] [12], which is a CNN for SCAMP-5, capable of MNIST digit recognition. Cain requires only 21 instructions whereas AUKE produces kernel code which has in total 49 instructions.…”

Section: Performance Evaluation Against Aukementioning

confidence: 99%

“…Cain requires only 21 instructions whereas AUKE produces kernel code which has in total 49 instructions. Reduced code not only improves the execution time, but also reduces the noise build up, which is significant problem as discussed in [20]. If the aim of finding sub-expressions is to eliminate redoing work, then the number of add and subtract operands is a proxy for how effective the search for sub-expressions is, regardless of how translations are handled.…”

Section: Performance Evaluation Against Aukementioning

confidence: 99%

See 3 more Smart Citations

Cain: Automatic Code Generation for Simultaneous Convolutional Kernels on Focal-plane Sensor-processors

Stow¹,

Murai²,

Saeedi³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Focal-plane Sensor-processors (FPSPs) are a camera technology that enable low power, high frame rate computation, making them suitable for edge computation. Unfortunately, these devices' limited instruction sets and registers make developing complex algorithms difficult. In this work, we present Cain 1 -a compiler that targets SCAMP-5, a general-purpose FPSP -which generates code from multiple convolutional kernels. As an example, given the convolutional kernels for an MNIST digit recognition neural network, Cain produces code that is half as long, when compared to the other available compilers for SCAMP-5.

show abstract

Section: Background: Scamp-5 Focal-plane Sensor-processormentioning

confidence: 99%

“…We will look at a simple example of how a convolutional kernel is represented in Cain. Here we use AnalogNet2 [20] [12] which is a CNN designed for SCAMP-5.…”

Section: Definitionsmentioning

confidence: 99%

Section: Performance Evaluation Against Aukementioning

confidence: 99%

Section: Performance Evaluation Against Aukementioning

confidence: 99%

See 2 more Smart Citations

Cain: Automatic Code Generation for Simultaneous Convolutional Kernels on Focal-plane Sensor-processors

Stow¹,

Murai²,

Saeedi³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Fortunately, we are on the brink of a new era of sensor design. The rise of programmable sensors [15], [16], [17], in which sensing and processing can be executed together…”

Section: Introductionmentioning

confidence: 99%

Learning Spatially Varying Pixel Exposures for Motion Deblurring

Nguyen¹,

Martel²,

Wetzstein³

2022

Preprint

View full text Add to dashboard Cite

Computationally removing the motion blur introduced by camera shake or object motion in a captured image remains a challenging task in computational photography. Deblurring methods are often limited by the fixed global exposure time of the image capture process. The post-processing algorithm either must deblur a longer exposure that contains relatively little noise or denoise a short exposure that intentionally removes the opportunity for blur at the cost of increased noise. We present a novel approach of leveraging spatially varying pixel exposures for motion deblurring using next-generation focal-plane sensor-processors along with an end-to-end design of these exposures and a machine learning-based motion-deblurring framework. We demonstrate in simulation and a physical prototype that learned spatially varying pixel exposures (L-SVPE) can successfully deblur scenes while recovering high frequency detail. Our work illustrates the promising role that focal-plane sensor-processors can play in the future of computational imaging.

show abstract

Compiling CNNs with Cain: focal-plane processing for robot navigation

Stow

Ahsan

et al. 2022

Auton Robot

View full text Add to dashboard Cite

Focal-plane Sensor-processors (FPSPs) are a camera technology that enables low power, high frame rate computation in the image sensor itself, making them suitable for edge computation. To fit into the sensor array, FPSPs are highly resource-constrained, with limited instruction set and few registers - which makes developing complex algorithms difficult. In this work, we present Cain, a compiler for convolutional filters that targets SCAMP-5, a general-purpose FPSP. Cain generates code to evaluate multiple convolutional kernels at the same time. It generates code that avoids the need for hardware multipliers, while orchestrating the exploitation of common sub-terms—leading to a large reduction in instruction count compared to both straightforward and prior optimized approaches. We demonstrate the capability enabled by Cain on SCAMP-5 with robotic navigation for near-sensor high-speed and low-power computation, by using Cain to implement a neural network on the focal plane.

show abstract

AnalogNet: Convolutional Neural Network Inference on Analog Focal Plane Sensor Processors

Cited by 7 publications

References 17 publications

Cain: Automatic Code Generation for Simultaneous Convolutional Kernels on Focal-plane Sensor-processors

Cain: Automatic Code Generation for Simultaneous Convolutional Kernels on Focal-plane Sensor-processors

Learning Spatially Varying Pixel Exposures for Motion Deblurring

Compiling CNNs with Cain: focal-plane processing for robot navigation

Contact Info

Product

Resources

About