A 1000 frames/s Vision Chip Using Scalable Pixel-Neighborhood-Level Parallel Processing

Schmitz, Joseph A.; Gharzai, Mahir Kabeer; Balkir, S.; Hoffman, Michael W.; White, Daniel J.; Schemm, Nathan

doi:10.1109/jssc.2016.2613094

Cited by 30 publications

(29 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently a novel vision chip based on pixel-neighborhood-level parallel processing was proposed [21]. The chip integrates a 2D array of neighborhood processors (NPs).…”

Section: Fd Vision Chipsmentioning

confidence: 99%

Neuromorphic vision chips

2018

Sci. China Inf. Sci.

View full text Add to dashboard Cite

The paper reviews the progress of neuromorphic vision chip research in decades. It focuses on two kinds of the neuromorphic vision chips: frame-driven (FD) and event-driven (ED) vision chips. The FD and ED vision chips are very different from each other in system architecture, image sensing, image information coding, image processing algorithm, design methodology. The vision chips can overcome serial data transmission and processing bottlenecks in traditional image processing systems. They can perform the high speed image capture and real-time image processing operations. This paper selects two typical chips from the two kinds of vision chips, respectively, and introduces their architectures, image sensing schemes, image processing processors and system operation. The FD neuromorphic reconfigurable vision chip comprises a high speed image sensor, a processing element array and self-organizing map neural network. The FD vision chip has the advantages in image resolution, static object detection, time-multiplex image processing, and chip area. The ED neuromorphic vision chip system is based on address-event-representation image sensor and event-driven multi-kernel convolution network. The ED vision chip has the advantages in fast sensing, low communication bandwidth, brain-like processing, and high energy efficiency. Finally, this paper discusses the architecture and the challenges of the future neuromorphic vision chip and indicates that the reconfigurable vision chip with left-and right-brain functions integrated in the three dimensional (3D) large-scale integrated circuit (LSI) technology becomes a trend of the research on the vision chip.

show abstract

“…Recently a novel vision chip based on pixel-neighborhood-level parallel processing was proposed [21]. The chip integrates a 2D array of neighborhood processors (NPs).…”

Section: Fd Vision Chipsmentioning

confidence: 99%

Neuromorphic vision chips

2018

Sci. China Inf. Sci.

View full text Add to dashboard Cite

show abstract

“…During the last decade several smart vision sensors have been designed in standard CIS technology [1,[3][4][5][6][7][8][9][10][11][12][13][14][15][16][17]. The increasing resolutions and frame rates result in a large data transfer between the imaging array and the processing unit.…”

Section: Vsocs State Of the Artmentioning

confidence: 99%

“…The straightforward approach is to implement in-pixel circuitry. Digital implementations offer high programmability: an example of the state of the art [3] performed edge detection, median filtering, histograms and tracking. But focal-plane digital circuits consume a lot of power and silicon area, especially through analog to digital converters (ADCs) [3,4].…”

Section: Vsocs State Of the Artmentioning

confidence: 99%

“…Digital implementations offer high programmability: an example of the state of the art [3] performed edge detection, median filtering, histograms and tracking. But focal-plane digital circuits consume a lot of power and silicon area, especially through analog to digital converters (ADCs) [3,4]. On the other hand, analog computations become more attractive for specialized tasks (implying restricted programmability) since analog operations can run faster at lower power.…”

Section: Vsocs State Of the Artmentioning

confidence: 99%

See 1 more Smart Citation

Distributed mixed-signal architecture for programmable smart image sensors

Hir

Kolar

Santos

2018

Analog Integr Circ Sig Process

View full text Add to dashboard Cite

Smart vision systems on a chip are promising for embedded applications. Currently, flexibility in the choice of integrated pre-processing tools is obtained at the expense of total silicon area and fill factor, which are otherwise optimized provided that the sensor performs a specific task. We propose a new architecture based on macropixel-level processing to improve the trade-off by using the same processing elements (PEs) for a whole group of pixels. In this paper, we show through transistor-level simulations the feasibility of using macropixel PEs. Their operative part is analog to avoid the bottleneck of analog to digital converters (ADC) and has digital control which is distributed in and out of the matrix of pixels. PEs are designed to be suitable for coefficient-reconfigurable spatial and temporal filtering. Sharing electronics among several pixels and matching existing algorithms to the target architecture allow for such programmability without degrading too much pixel area nor fill factor.

show abstract

“…Whenever one pixel is fed from the sensing component, the processing blocks in the core are invoked, and they operate on the pixel data in a pipeline scheme, and then output one pixel down the stream. Compared to the pixel- or column-parallel processing components commonly used in previous smart image sensors [10], [11], the proposed streaming architecture has three advantages. First, it is ready to be integrated with standard image sensors, since common image sensors output pixels in a serial stream, which directly fits our magnification core.…”

Section: Introductionmentioning

confidence: 99%

A Streaming Motion Magnification Core for Smart Image Sensors

Shi

Luo

2018

IEEE Trans. Circuits Syst. II

View full text Add to dashboard Cite

This paper proposes a modified Eulerian Video Magnification (EVM) algorithm and a hardware implementation of a motion magnification core for smart image sensors. Compared to the original EVM algorithm, we perform the pixel-wise temporal bandpass filtering only once rather than multiple times on all scale layers, to reduce the memory and multiplier requirement for hardware implementation. A pixel stream processing architecture with pipelined blocks is proposed for the magnification core, enabling it to readily fit common image sensing components with streaming pixel output, while achieving higher performance with lower system cost. We implemented an FPGA-based prototype that is able to process up to 90M pixels per second and magnify subtle motion. The motion magnification results are comparable to the original algorithm running on PC.

show abstract

A 1000 frames/s Vision Chip Using Scalable Pixel-Neighborhood-Level Parallel Processing

Cited by 30 publications

References 35 publications

Neuromorphic vision chips

Neuromorphic vision chips

Distributed mixed-signal architecture for programmable smart image sensors

A Streaming Motion Magnification Core for Smart Image Sensors

Contact Info

Product

Resources

About