An embedded face-classification system for infrared images on an FPGA

Abstract

We present a face-classification architecture for long-wave infrared (IR) images implemented on a Field Programmable Gate Array (FPGA). The circuit is fast, compact and low power, can recognize faces in real time and be embedded in a larger image-processing and computer vision system operating locally on an IR camera. The algorithm uses Local Binary Patterns (LBP) to perform feature extraction on each IR image. First, each pixel in the image is represented as an LBP pattern that encodes the similarity between the pixel and its neighbors. Uniform LBP codes are then used to reduce the number of patterns to 59 while preserving more than 90% of the information contained in the original LBP representation. Then, the image is divided into 64 non-overlapping regions, and each region is represented as a 59-bin histogram of patterns. Finally, the algorithm concatenates all 64 regions to create a 3,776-bin spatially enhanced histogram. We reduce the dimensionality of this histogram using Linear Discriminant Analysis (LDA), which improves clustering and enables us to store an entire database of 53 subjects on-chip. During classification, the circuit applies LBP and LDA to each incoming IR image in real time, and compares the resulting feature vector to each pattern stored in the local database using the Manhattan distance. We implemented the circuit on a Xilinx Artix-7 XC7A100T FPGA and tested it with the UCHThermalFace database, which consists of 28 81 ⇥ 150-pixel images of 53 subjects in indoor and outdoor conditions. The circuit achieves a 98.6% hit ratio, trained with 16 images and tested with 12 images of each subject in the database. Using a 100 MHz clock, the circuit classifies 8,230 images per second, and consumes only 309mW.

Face Recognition on a Smart Image Sensor Using Local Gradients

“…It uses a Virtex-6 LX550T FPGA to process 450 pixel images per second. Soto et al [ 49 ] proposed an embedded face classification circuit for IR images on an FPGA that uses LBP and linear discriminant analysis (LDA), achieves 98.6% accuracy using a thermal IR database of 53 subjects, and can classify 8230 images per second with a power consumption of 309 mW.…”

Embedded nonuniformity correction in infrared focal plane arrays using the Constant Range algorithm

“…Because typical NUC algorithms are computationally expensive, the embedded hardware executing the algorithm must provide high performance while operating under severe restrictions in power, size, and cost. In many applications, common programmable devices such as microprocessors and digital signal processors (DSPs) cannot deliver the performance needed by the application under these restrictions, especially if the device must also perform higher-level video processing tasks such as super resolution [5,6], video stabilization [7,8], face detection [9,10]. In these cases, dedicated hardware solutions may be needed in order to achieve the performance/power/size/cost tradeoff required by the application.…”