Bottom‐up performance analysis of focal‐plane mixed‐signal hardware for Viola–Jones early vision tasks

Abstract

SUMMARYFocal-plane mixed-signal arrays have traditionally been designed according to the general claim that moderate accuracy in processing is affordable. The performance of their circuitry has been analyzed in these terms without a comprehensive study of the ultimate consequences of such moderate accuracy. In this paper, for the first time to the best of our knowledge, we do carry out this study. We move expectable performance of mixed-signal image processing hardware directly into the vision algorithm making use of it. This permits to close a wider design loop, enabling a more aggressive design of this kind of hardware provided that the algorithm, at the highest level-semantic interpretation of the scene-, can afford it. Thus, we present a thorough analysis of the non-idealities associated with the implementation of a QVGA array tailored for the distinctive characteristics of the Viola-Jones processing framework. The resulting deviation models are then introduced in the processing flow of this framework provided by the OpenCV library. We have found, contrary to what could be expected, that these deviations do not necessarily degrade the performance of the Viola-Jones algorithm. They could be even beneficial for certain high-level specifications. Additionally, we demonstrate the architectural advantages of our approach: exploitation of focal-plane distributed memory and ultra-low-power operation.

CMOS Vision Sensors: Embedding Computer Vision at Imaging Front-Ends

“…This stored voltage at this node will be employed later to evaluate the average value of different neighbourhoods. The array can be divided into different regions by means of control lines distributed along the horizontal and vertical edges of the array [55], which are operated by peripheral control blocks and selection registers. These registers can be serially updated with different interconnection patterns.…”

“…Functional diagram of the chip architecture and schematic of the pixel of a CVIS for privacy-aware applications[19] [55].…”

Image Feature Extraction Acceleration

“…Each diffusion process producing an integral image pixel is followed by a stage of analog-to-digital conversion that takes place concurrently with the readjustment of the interconnection patterns for the next pixel to be computed. More details about the whole process and the additional circuitry required per pixel can be found in [34]. …”

“…There is also the possibility of loading in parallel up to six different patterns representing six successive image pixelation scales. This is achieved by means of control signals distributed regularly along the horizontal and vertical dimensions of the array [34]. The reconfiguration signals coming from the periphery map into the signals EN S i,i+1 , EN S j,j+1 , EN SQ i,i+1 and EN SQ j,j+1 at pixel level, where the coordinates (i, j) denote the location of the array cell considered.…”

Focal-Plane Sensing-Processing: A Power-Efficient Approach for the Implementation of Privacy-Aware Networked Visual Sensors

“…There is also the possibility of loading in parallel up to six different patterns representing six successive image pixelation scales. This is achieved by means of control signals distributed regularly along the horizontal and vertical dimensions of the array [31]. The reconfiguration signals coming from the periphery map into the signals EN S i,i +1 , EN S j,j +1 , $\overline{{\text{EN}}_{S{Q}_{i,j+1}}}$ and $\overline{{\text{EN}}_{S{Q}_{j,j+1}}}$ at pixel level, where the coordinates ( i , j ) denote the location of the array cell considered.…”