A 464GOPS 620GOPS/W heterogeneous multi-core SoC for image-recognition applications

Tanabe, Yoshinori; Sumiyoshi, M.; Nishiyama, Masayoshi; Yamazaki, Isamu; Fujii, Shinsuke; Kimura, Kozo; Aoyama, Takuma; Banno, Moriyasu; Hayashi, H.; Miyamori, Takashi

doi:10.1109/isscc.2012.6176984

Cited by 25 publications

(13 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, no single feature-classifier pair has been reported in literature which is better discriminant as well as computationally less complex than HOG as noted by Dollar et al [8], Rodrigo et al [15], Achmad et al [42] and Yong et al [37]. Hence, in the recent years many dedicated hardware designs for object detection have adopted HOG-Linear SVM despite its computational complexity [53][54][55][56][57][58][59][60]. To this end, in the next section we propose a low complexity pedestrian detection framework which is well suited for small embedded systems without GPU/SSE support and requires minimal hardware resources when implemented on an FPGA.…”

Section: Real-time Performance Of Pedestrian Detectorsmentioning

confidence: 99%

A Low-Complexity Pedestrian Detection Framework for Smart Video Surveillance Systems

Bilal

Khan

et al. 2017

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Pedestrian detection is a key problem in computer vision and is currently addressed with increasingly complex solutions involving compute-intensive features and classification schemes. In this scope, Histogram of Oriented Gradients (HOG) in conjunction with linear SVM classifier is considered to be the single most discriminative feature which has been adopted as a stand-alone detector as well as a key instrument in advance systems involving hybrid features and cascaded detectors. In this paper, we propose a pedestrian detection framework which is computationally less expensive as well as more accurate than HOG-linear SVM. The proposed scheme exploits the discriminating power of the locally significant gradients in building orientation histograms without involving complex floating point operations while computing the feature. The integeronly feature allows the use of powerful Histogram Intersection Kernel SVM classifier in a fast look-up table based implementation. Resultantly, the proposed framework achieves at least 3% more accurate detection results than HOG on standard datasets while being 1.8 and 2.6 times faster on conventional desktop PC and embedded ARM platforms respectively for a single scale pedestrian detection on VGA resolution video. Additionally, hardware implementation on Altera Cyclone IV FPGA results in more than 40% savings in logic resources compared to its HOG-Linear SVM competitor. Hence, the proposed feature and classification setup is shown to be a better candidate as the single most discriminative pedestrian detector than currently accepted HOG-linear SVM.

show abstract

Section: Real-time Performance Of Pedestrian Detectorsmentioning

confidence: 99%

A Low-Complexity Pedestrian Detection Framework for Smart Video Surveillance Systems

Bilal

Khan

et al. 2017

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

show abstract

“…Image recognition ICs for an advanced driver assistance system (ADAS) have also been proposed [3]. However, future ADAS applications must support greater numbers of real-time recognition processes simultaneously, with higher detection rates and lower false-positive rates.…”

mentioning

confidence: 99%

“…Two 64b LPDDR2 I/Fs provide 12.8GB/s bandwidth. In order to improve flexibility, two low-power 4-core processor clusters [3] are integrated, based on a VLIW architecture with a SIMD instruction set. Each processor core can execute up to three slots of a VLIW instruction every cycle.…”

mentioning

confidence: 99%

“…These instructions can be executed with an initiation interval of one cycle. The following eight hard-wired accelerators are included to enhance the performance of image recognition processes: 1) a CoHOG (co-occurrence histograms of oriented gradients) accelerator [3][4] for object classification; 2) a histogram accelerator for making histograms from images; 3) two 64-parallel SIMD filter accelerators for various filtering tasks, such as noise reduction and gradient-image generation; 4) three affine accelerators for image transformation and lens calibration tasks; 5) two matching accelerators for stereo matching and tracking tasks; 6) two pyramid accelerators for making pyramid images; 7) a structure-from-motion (SfM) accelerator; and, 8) two accelerators called HOX for color-based object classification. Because accelerators can execute tasks in a fixed number of cycles, it eases real-time processing versus employing a processor-based solution.…”

mentioning

confidence: 99%

“…In the color mode, it handles four color-based feature descriptors, and it calculates four likelihood scores from four dictionaries of four color-based feature descriptors simultaneously, with 1 pixel/cycle throughput. The CoHOG accelerator in previous work [3] calculates a likelihood score from a dictionary with 1 pixel/cycle throughput. Therefore, the HOX accelerator achieves higher throughput than the CoHOG accelerator: eight times higher in the CoHOG mode, and four times higher in the color mode.…”

mentioning

confidence: 99%

See 2 more Smart Citations

18.2 A 1.9TOPS and 564GOPS/W heterogeneous multicore SoC with color-based object classification accelerator for image-recognition applications

Tanabe

Sano

Yamada

et al. 2015

2015 IEEE International Solid-State Circuits Conference - (ISSCC) Digest of Technical Papers

Self Cite

View full text Add to dashboard Cite

Image recognition technologies have gained prominence in a variety of fields, such as automotive and surveillance, with dedicated image-recognition ICs being developed recently [1][2]. Image recognition ICs for an advanced driver assistance system (ADAS) have also been proposed [3]. However, future ADAS applications must support greater numbers of real-time recognition processes simultaneously, with higher detection rates and lower false-positive rates. For instance, adaptive cruise control (ACC), an application of ADAS, comprises many image recognition processes, such as pedestrian detection (PD), vehicle detection (VD), general obstacle detection (GOD), lane detection (LD), traffic light recognition (TLR), and traffic sign recognition (TSR). ACC also requires high detection accuracy to prevent unnecessary braking or acceleration. To satisfy these requirements, we have developed an SoC with two 4-core processor clusters and 14 hard-wired accelerators. It is designed to realize the six recognition processes (PD, VD, GOD, LD, TLR, and TSR) for ACC and automatic high beam (AHB) for headlight control. It achieves 1.9TOPS peak performance in 3.37W. This low power consumption enables the SoC to operate with passive cooling in a high-temperature automotive environment. Figure 18.2.1 shows a block diagram of the SoC. It comprises two 4-core processor clusters and 14 hard-wired accelerators of which there are eight variants, two RISC cores, 1.5MB on-chip SRAM, two LPDDR2 interfaces (I/F), four video inputs, and two video outputs. Considering the product lifespan and high quality requirements of automotive systems, we select LPDDR2 instead of LPDDR3. Two 64b LPDDR2 I/Fs provide 12.8GB/s bandwidth. In order to improve flexibility, two low-power 4-core processor clusters [3] are integrated, based on a VLIW architecture with a SIMD instruction set. Each processor core can execute up to three slots of a VLIW instruction every cycle. As floating point is becoming essential in image recognition, we have added a double precision floating-point execution unit to each processor. Each processor core can execute up to two SIMD integer instructions (where each instruction is 8-parallel 8b) or two double-precision floating-point instructions, such as addition and multiplication instructions. These instructions can be executed with an initiation interval of one cycle. The following eight hard-wired accelerators are included to enhance the performance of image recognition processes: 1) a CoHOG (co-occurrence histograms of oriented gradients) accelerator [3-4] for object classification; 2) a histogram accelerator for making histograms from images; 3) two 64-parallel SIMD filter accelerators for various filtering tasks, such as noise reduction and gradient-image generation; 4) three affine accelerators for image transformation and lens calibration tasks; 5) two matching accelerators for stereo matching and tracking tasks; 6) two pyramid accelerators for making pyramid images; 7) a structure-from-motion (SfM) accelerator; and, 8) two accel...

show abstract

A Real-Time and Energy-Efficient Embedded System for Intelligent ADAS with RNN-Based Deep Risk Prediction using Stereo Camera

Lee

Choe

Bong

et al. 2017

Lecture Notes in Computer Science

View full text Add to dashboard Cite

A 464GOPS 620GOPS/W heterogeneous multi-core SoC for image-recognition applications

Cited by 25 publications

References 4 publications

A Low-Complexity Pedestrian Detection Framework for Smart Video Surveillance Systems

A Low-Complexity Pedestrian Detection Framework for Smart Video Surveillance Systems

18.2 A 1.9TOPS and 564GOPS/W heterogeneous multicore SoC with color-based object classification accelerator for image-recognition applications

A Real-Time and Energy-Efficient Embedded System for Intelligent ADAS with RNN-Based Deep Risk Prediction using Stereo Camera

Contact Info

Product

Resources

About