Towards an Efficient Hardware Implementation of CNN-Based Object Trackers

Abstract

Convolution Neural Network (CNN) has achieved a performance boost to the visual tracking field. However, CNN-based trackers feature slow computational speed and large memory size. These issues challenge the embedded implementation of the CNN-based trackers. In this paper, we show how interpolation schemes can significantly reduce the memory requirements. In addition, we present a design-space exploration of the fixed-point representation of the CNN-based trackers aiming for a cost-efficient hardware implementation. Moreover, we present a hardware accelerator of the online training stage of the tracker. Our proposed accelerator can train the whole fully connected layers at a rate of 45.9 frames-per-second.

An efficient hardware implementation of CNN-based object trackers for real-time applications

“…We study the performance impact if we remove or approximate these features in addition to the gain that can be achieved in return. In [14], we explored the design space of the fixed-point representation of the main parameters of ILNET and we studied 11 ILNET variants using OTB-100 benchmark.…”

“…Although we showed in [14] that the fixed-point representation of the fully connected layers can be reduced down to 13-bit and 12-bit for the weights and the feature maps, respectively, with small performance degradation, we adopt 16-bit in this section because the external memories are typically byte-aligned and this 16-bit representation simplifies the memory data allocation and data access.…”

“…In addition, our proposed training scheme features a constant training time even in case of the frames that require a short-term update. We analyzed the speed of ILNET5 [14] and all the ILNET variants presented in this paper by running the Matlab system model on all OTB-100 videos using an Intel i7-3520 M CPU system. ILNET5 runs in 0.78 s per frame, while ILNET17 run in 0.54 s per frame.…”

“…Initial results of this work were published in [12,14]. In [12], we outlined the overall architecture and presented a Matlab model of our proposed object tracker named Interpolation and Localization Network (ILNET).…”

“…In [12], we outlined the overall architecture and presented a Matlab model of our proposed object tracker named Interpolation and Localization Network (ILNET). Reference [14] presented an initial HW design of ILNET where we showed the benefits of applying interpolation schemes and word length exploration of the different variables to cut-down the required memory size and speed-up the processing time. We presented also a novel technique for the error back-propagation that avoids the transposed reading from the external memory.…”