Real-time background generation and foreground object segmentation for high-definition colour video stream in FPGA device

Kryjak, Tomasz; Komorkiewicz, Mateusz; Gorgon, Marek Boguslaw

doi:10.1007/s11554-012-0290-5

Cited by 30 publications

(6 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a Full HD image has a considerable size, the external DDR3 memory is used for this task. Detailed description of the memory controller is presented in the article [ 49 ].…”

Section: System Verificationmentioning

confidence: 99%

Efficient Hardware Implementation of the Horn-Schunck Algorithm for High-Resolution Real-Time Dense Optical Flow Sensor

Komorkiewicz

Kryjak

Gorgon

2014

Sensors

Self Cite

View full text Add to dashboard Cite

This article presents an efficient hardware implementation of the Horn-Schunck algorithm that can be used in an embedded optical flow sensor. An architecture is proposed, that realises the iterative Horn-Schunck algorithm in a pipelined manner. This modification allows to achieve data throughput of 175 MPixels/s and makes processing of Full HD video stream (1, 920 × 1, 080 @ 60 fps) possible. The structure of the optical flow module as well as pre- and post-filtering blocks and a flow reliability computation unit is described in details. Three versions of optical flow modules, with different numerical precision, working frequency and obtained results accuracy are proposed. The errors caused by switching from floating- to fixed-point computations are also evaluated. The described architecture was tested on popular sequences from an optical flow dataset of the Middlebury University. It achieves state-of-the-art results among hardware implementations of single scale methods. The designed fixed-point architecture achieves performance of 418 GOPS with power efficiency of 34 GOPS/W. The proposed floating-point module achieves 103 GFLOPS, with power efficiency of 24 GFLOPS/W. Moreover, a 100 times speedup compared to a modern CPU with SIMD support is reported. A complete, working vision system realized on Xilinx VC707 evaluation board is also presented. It is able to compute optical flow for Full HD video stream received from an HDMI camera in real-time. The obtained results prove that FPGA devices are an ideal platform for embedded vision systems.

show abstract

“…As a Full HD image has a considerable size, the external DDR3 memory is used for this task. Detailed description of the memory controller is presented in the article [ 49 ].…”

Section: System Verificationmentioning

confidence: 99%

Efficient Hardware Implementation of the Horn-Schunck Algorithm for High-Resolution Real-Time Dense Optical Flow Sensor

Komorkiewicz

Kryjak

Gorgon

2014

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the work [19] a background generation method based on the Clustering algorithm and foreground object segmentation using information about intensity, colour and texture was presented. The system was working with CIE Lab colour space.…”

Section: A Related Workmentioning

confidence: 99%

“…In previous works a special controller (MEM CTRL) was designed to perform a sequential read and write of a 1024-bit vector. Details are available in [19].…”

Section: B Overview Of the Systemmentioning

confidence: 99%

Real-time Foreground Object Detection Combining the PBAS Background Modelling Algorithm and Feedback from Scene Analysis Module

Kryjak¹,

Komorkiewicz²,

Gorgon³

2014

International Journal of Electronics and Telecommunications

Self Cite

View full text Add to dashboard Cite

Abstract-The article presents a hardware implementation of the foreground object detection algorithm PBAS (Pixel-Based Adaptive Segmenter) with a scene analysis module. A mechanism for static object detection is proposed, which is based on consecutive frame differencing. The method allows to distinguish stopped foreground objects (e.g. a car at the intersection, abandoned luggage) from false detections (so-called ghosts) using edge similarity. The improved algorithm was compared with the original version on popular test sequences from the changedetection.net dataset. The obtained results indicate that the proposed approach allows to improve the performance of the method for sequences with the stopped objects. The algorithm has been implemented and successfully verified on a hardware platform with Virtex 7 FPGA device. The PBAS segmentation, consecutive frame differencing, Sobel edge detection and advanced one-pass connected component analysis modules were designed. The system is capable of processing 50 frames with a resolution of 720 × 576 pixels per second.

show abstract

“…Chang et al 8 proposed a high-speed Harris corner detector on FPGA device, with high efficiency and high frame rate (540 fps). Kryjak et al 9 realized a high-resolution image-processing platform based on FPGA; their background generation and target detection algorithm were of high efficiency and robustness. Anderson et al 10 proposed an FPGA-based vision system for an autonomous mobile robot, including processes of object detection, tracking, and path planning.…”

Section: Introductionmentioning

confidence: 99%

Real-time object tracking system based on field-programmable gate array and convolution neural network

Lyu

Chen

Jiang

et al. 2016

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

Vision-based object tracking has lots of applications in robotics, like surveillance, navigation, motion capturing, and so on. However, the existing object tracking systems still suffer from the challenging problem of high computation consumption in the image processing algorithms. The problem can prevent current systems from being used in many robotic applications which have limitations of payload and power, for example, micro air vehicles. In these applications, the central processing unit-or graphics processing unit-based computers are not good choices due to the high weight and power consumption. To address the problem, this article proposed a real-time object tracking system based on fieldprogrammable gate array, convolution neural network, and visual servo technology. The time-consuming image processing algorithms, such as distortion correction, color space convertor, and Sobel edge, Harris corner features detector, and convolution neural network were redesigned using the programmable gates in field-programmable gate array. Based on the field-programmable gate array-based image processing, an image-based visual servo controller was designed to drive a two degree of freedom manipulator to track the target in real time. Finally, experiments on the proposed system were performed to illustrate the effectiveness of the real-time object tracking system.

show abstract

Real-time background generation and foreground object segmentation for high-definition colour video stream in FPGA device

Cited by 30 publications

References 33 publications

Efficient Hardware Implementation of the Horn-Schunck Algorithm for High-Resolution Real-Time Dense Optical Flow Sensor

Efficient Hardware Implementation of the Horn-Schunck Algorithm for High-Resolution Real-Time Dense Optical Flow Sensor

Real-time Foreground Object Detection Combining the PBAS Background Modelling Algorithm and Feedback from Scene Analysis Module

Real-time object tracking system based on field-programmable gate array and convolution neural network

Contact Info

Product

Resources

About