Real-time low-power neuromorphic hardware for autonomous object recognition

Khosla, Deepak; Chen, Yang; Huber, David; Buer, Darrel J. Van; Kim, Kyungnam; Cheng, Shinko Y.

doi:10.1117/12.2015944

Cited by 5 publications

(3 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…A team led by HRL Laboratories, LLC (HRL) included participants from New York University, Johns Hopkins University, MIT, Brown University, Yale University, Argon ST and Imagize, LLC. Our team successfully developed a realtime, accurate, and low-power neuromorphic solution for automated object recognition in images and videos [2,3]. Our system called NIVA (Neuromorphic Image and Video Analysis) is an autonomous object recognition system based on a visual cognition architecture ( Fig.…”

Section: A Hrl's Neuromorphic Image and Video Analysis (Niva)mentioning

confidence: 99%

Performance Evaluation of Neuromorphic-Vision Object Recognition Algorithms

Kasturi

Goldgof

Ekambaram

et al. 2014

2014 22nd International Conference on Pattern Recognition

Self Cite

View full text Add to dashboard Cite

The U.S. Defense Advanced Research Projects Agency's (DARPA) Neovision2 program aims to develop artificial vision systems based on the design principles employed by mammalian vision systems. Three such algorithms are briefly described in this paper. These neuromorphic-vision systems' performance in detecting objects in video was measured using a set of annotated clips. This paper describes the results of these evaluations including the data domains, metrics, methodologies, performance over a range of operating points and a comparison with computer vision based baseline algorithms.

show abstract

Section: A Hrl's Neuromorphic Image and Video Analysis (Niva)mentioning

confidence: 99%

Performance Evaluation of Neuromorphic-Vision Object Recognition Algorithms

Kasturi

Goldgof

Ekambaram

et al. 2014

2014 22nd International Conference on Pattern Recognition

Self Cite

View full text Add to dashboard Cite

show abstract

“…The NEOVUS is implemented in COTS hardware to achieve real-time performance at low size, weight, and power (SWaP). Several components and capabilities of NEOVUS have been previously described by us Chen et al ( 2011 ), Khosla et al ( 2013a ), Khosla et al ( 2013b ), Honda et al ( 2013 ). This paper describes the complete system end-to-end, provides additional details of components and key capabilities, and describes in detail the results of DARPA evaluation on urban datasets.…”

Section: Introductionmentioning

confidence: 99%

A neuromorphic system for video object recognition

Khosla

Chen

Kim

2014

Front. Comput. Neurosci.

Self Cite

View full text Add to dashboard Cite

Automated video object recognition is a topic of emerging importance in both defense and civilian applications. This work describes an accurate and low-power neuromorphic architecture and system for real-time automated video object recognition. Our system, Neuormorphic Visual Understanding of Scenes (NEOVUS), is inspired by computational neuroscience models of feed-forward object detection and classification pipelines for processing visual data. The NEOVUS architecture is inspired by the ventral (what) and dorsal (where) streams of the mammalian visual pathway and integrates retinal processing, object detection based on form and motion modeling, and object classification based on convolutional neural networks. The object recognition performance and energy use of the NEOVUS was evaluated by the Defense Advanced Research Projects Agency (DARPA) under the Neovision2 program using three urban area video datasets collected from a mix of stationary and moving platforms. These datasets are challenging and include a large number of objects of different types in cluttered scenes, with varying illumination and occlusion conditions. In a systematic evaluation of five different teams by DARPA on these datasets, the NEOVUS demonstrated the best performance with high object recognition accuracy and the lowest energy consumption. Its energy use was three orders of magnitude lower than two independent state of the art baseline computer vision systems. The dynamic power requirement for the complete system mapped to commercial off-the-shelf (COTS) hardware that includes a 5.6 Megapixel color camera processed by object detection and classification algorithms at 30 frames per second was measured at 21.7 Watts (W), for an effective energy consumption of 5.45 nanoJoules (nJ) per bit of incoming video. These unprecedented results show that the NEOVUS has the potential to revolutionize automated video object recognition toward enabling practical low-power and mobile video processing applications.

show abstract

“…Outras aplicações incluem o uso de DSPs, como em [26], no qual um sistema de classificação de imagens foi embarcado para a detecção automática de pequenos defeitos na confecção têxtil. Além disso, FPGAs têm sido também utilizados para embarcar algoritmos do estado da arte da classificação de imagens, como em [17], no qual os autores embarcaram uma rede neural convolucional (CNN, do inglês convolutional neural network ) [19] para detecção de objetos, resultando em um desempenho que permitiu classificar vídeo em tempo real. Entretanto, um ponto negativo dessas abordagens é a necessidade do uso de métodos especializados para a extração e/ou seleção de características, geralmente baseadas em métodos genéricos como transformada discreta de cossenos (DCT, do inglês discrete cosine transform), transformada discreta de Fourier (DFT, do inglês discrete Fourier transform), banco de filtros, Wavelets, ou mesmo exigindo conhecimento de um especialista na fase de pré-processamento [12].…”

Section: Contextualizaçãounclassified

Redução de custo computacional em classificações baseadas em transformadas aprendidas

Machado¹

View full text Add to dashboard Cite

We present a theoretical analysis and empirical evaluations of a novel set of techniques for computational cost reduction of classifiers that are based on learned transform and soft-threshold. By modifying optimization procedures for dictionary and classifier training, as well as the resulting dictionary elements, our techniques allow to reduce the bit precision and to replace each floating-point multiplication by a single integer bit shift. We also show how the optimization algorithms in some dictionary training methods can be modified to penalize higher-energy dictionaries. We applied our techniques with the classifier Learning Algorithm for Soft-Thresholding, testing on the datasets used in its original paper. Our results indicate it is feasible to use solely sums and bit shifts of integers to classify at test time with a limited reduction of the classification accuracy. These low power operations are a valuable trade off in FPGA implementations as they increase the classification throughput while decrease both energy consumption and manufacturing cost. Moreover, our techniques reduced almost 50% of the bit precision in almost all datasets we tested.

show abstract

Real-time low-power neuromorphic hardware for autonomous object recognition

Cited by 5 publications

References 2 publications

Performance Evaluation of Neuromorphic-Vision Object Recognition Algorithms

Performance Evaluation of Neuromorphic-Vision Object Recognition Algorithms

A neuromorphic system for video object recognition

Redução de custo computacional em classificações baseadas em transformadas aprendidas

Contact Info

Product

Resources

About