Efficient Hardware Implementation of the Horn-Schunck Algorithm for High-Resolution Real-Time Dense Optical Flow Sensor

Abstract

This article presents an efficient hardware implementation of the Horn-Schunck algorithm that can be used in an embedded optical flow sensor. An architecture is proposed, that realises the iterative Horn-Schunck algorithm in a pipelined manner. This modification allows to achieve data throughput of 175 MPixels/s and makes processing of Full HD video stream (1, 920 × 1, 080 @ 60 fps) possible. The structure of the optical flow module as well as pre- and post-filtering blocks and a flow reliability computation unit is described in details. Three versions of optical flow modules, with different numerical precision, working frequency and obtained results accuracy are proposed. The errors caused by switching from floating- to fixed-point computations are also evaluated. The described architecture was tested on popular sequences from an optical flow dataset of the Middlebury University. It achieves state-of-the-art results among hardware implementations of single scale methods. The designed fixed-point architecture achieves performance of 418 GOPS with power efficiency of 34 GOPS/W. The proposed floating-point module achieves 103 GFLOPS, with power efficiency of 24 GFLOPS/W. Moreover, a 100 times speedup compared to a modern CPU with SIMD support is reported. A complete, working vision system realized on Xilinx VC707 evaluation board is also presented. It is able to compute optical flow for Full HD video stream received from an HDMI camera in real-time. The obtained results prove that FPGA devices are an ideal platform for embedded vision systems.

An FPGA-optimized architecture of horn and schunck optical flow algorithm for real-time applications

“…In contrast, the benefit of using parallelism due to pipelining is illustrated by Komorkiewicz et al [8]. Although the approach is similar to the parallel approach which is presented here, the performance is worse.…”

Realization of 3D Aqua Hologram Augmented Reality of Robot Fish

“…Its operation is entirely based on the video frames of moving objects. (7,8) In our proposed aquarium for displaying 3D holograms of robot fish, to make the captured video smoother, we used the optical flow algorithm shown in Fig. 2.…”

Real-time hardware–software embedded vision system for ITS smart camera implemented in Zynq SoC

“…Furthermore, the study [20] showed that pipelined image processing systems can achieve linear acceleration. Outstanding computing performances of parallel-pipelined modules for the Horn-Schunck optical flow algorithm were demonstrated in papers [31,35].…”

“…First, because it is consistent with the data sending method from the image sensor, and therefore frame buffering between successive operations is not required. Second, as already mentioned, the linear acceleration in a pipeline system [20] allows to achieve very high computing performance (operations per second) and energy efficiency (operations per watt) [31].…”

“…In some cases, it is necessary to temporary cache the full image frame. Examples include two-pass connected component labelling [27], optical flow computation [31] or foreground object segmentation [34]. Nevertheless, the data temporarily stored in the buffer are transmitted again to the rest of the system in the form of a video stream.…”