A 345mW heterogeneous many-core processor with an intelligent inference engine for robust object recognition

Abstract

Abstract-A heterogeneous many-core object recognition processor is proposed to realize robust and efficient object recognition on real-time video of cluttered scenes. Unlike previous approaches that simply aimed for high GOPS/W, we aim to achieve high Effective GOPS/W, or EGOPS/W, which only counts operations carried out on meaningful regions of an input image. This is achieved by the Unified Visual Attention Model (UVAM) which confines complex Scale Invariant Feature Transform (SIFT) feature extraction to meaningful object regions while rejecting meaningless background regions. The Intelligent Inference Engine (IIE), a mixed-mode neuro-fuzzy inference system, performs the top-down familiarity attention of the UVAM which guides attention toward pre-learned objects. Weight perturbation-based learning of the IIE ensures high attention precision through online adaptation. The SIFT recognition is accelerated by an optimized array of 4 20-way SIMD Vector Processing Elements, 32 MIMD Scalar Processing Elements, and 1 Feature Matching Processor. When processing 30 fps 640 480 video, the 50 mm 2 object recognition processor implemented in a 0.13 m process achieves 246 EGOPS/W, which is 46% higher than the previous work. The average power consumption is only 345 mW.

A 52 mW Full HD 160-Degree Object Viewpoint Recognition SoC With Visual Vocabulary Processor for Wearable Vision Applications

“…The peak performance is 165GOPS while the average power efficiency is 1.18TOPS/W. The area efficiency of 25.9GOPS/mm 2 is 2× better than the previous works [2] [3]. Overall system comparison is listed in Fig.…”

“…Lastly, VVP is designed to simplify the complicated computation. Existing object recognition systems [1][2][3] operate object recognition in feature matching stage and require frequent memory accesses. More memory access leads to higher power consumption that is critical in wearable applications.…”

1.15mW mixed-mode neuro-fuzzy accelerator for keypoint localization in image processing

“…The proposed neuro-fuzzy accelerator achieved 1.15mW power at 1.2V and 0.733mm 2 area consumption. Compared to typical heterogeneous architecture of processing elements with 1 SIMD PE and 8 MIMD PEs [3] the proposed architecture achieved 43% reduction of localization processing time, and 19% reduction for image feature extraction time at the expense of additional 1.15m W additional power consumption as shown in Fig. 8 For keypoints localization in SIFT algorithm, mixed mode neuro-fuzzy accelerator is proposed.…”

A 92mW real-time traffic sign recognition system with robust light and dark adaptation

“…Object recognition systems are the topic of interest in various fields, such as intelligent robots, security, and automotive vision systems [1][2][3]. Among the applications, the driver assistant system has been emerging as an important area because the traffic problem gets more and more serious.…”