Real‐time multiview human pose tracking using graphics processing unit‐accelerated particle swarm optimization

Abstract

This paper describes how to achieve real-time tracking of 3D human motion using multiview images and graphics processing unit (GPU)-accelerated particle swarm optimization. The tracking involves configuring the 3D human model in the pose described by each particle and then rasterizing it in each 2D plane. The Compute Unified Device Architecture threads rasterize the columns of the triangles and perform the summing of the fitness values of pixels belonging to the processed columns. Such a parallel particle swarm optimization (PSO) exhibits the level of parallelism that allows us to effectively utilize the GPU resources. Image acquisition and image processing are multithreaded and run on CPU in parallel with PSO-based searching for the best pose. Owing to such task decomposition, the tracking of the full human body can be performed at rates of 12 frames per second. For a PSO consisting of 1000 particles and executing 10 iterations, the GPU achieves an average speedup of 12 over the CPU. Using marker-less motion capture system consisting of four calibrated and synchronized cameras, the efficiency comparisons were conducted on four CPU cores and four GTX GPUs on two cards. 1552 B. RYMUT AND B. KWOLEK icant speedups in comparison with homogeneous multicore systems in the same price range. These papers initiated a passionate debate on the limits of GPU acceleration for various classes of applications [2]. A comparison of 14 various implementations showed speedups from 0:5 to 15 (GPU over CPU). There is a common agreement that in order to achieve such speedups, the algorithms to be executed on GPU should be carefully designed. Even though it is easier to program GPUs than ever, efficiently taking advantages of GPU resources still requires unique techniques.The CPUs are still the most frequently used hardware for image processing, given their versatility and tremendous speed. On the other hand, image processing algorithms are good candidates for GPU implementation, because many image processing operations have high inherent parallelism, which is frequently achieved through per-pixel operations. Many research reports confirmed this by showing GPU acceleration of many image processing algorithms [3]. A recent study [1] reports a 30-fold speedup for low-level algorithms and up to 10-fold speedup for high-level functions, which contain more overhead and many steps that are not easy to parallelize.Non-intrusive human pose tracking is a key issue in computer vision because of many possible applications in surveillance, human activity recognition, virtual reality, motion capture, and so on. Because of variations in individual body shapes, this is one of the most challenging problems in computer vision being at the same time one of the most computationally demanding tasks. Particle filtering is widely used in articulated motion tracking because of its ability to represent multimodal probability distributions and to maintain multiple pose hypotheses. Several improvements of ordinary particle filter (PF) were proposed to achieve f...

Reconstruction of 3D human motion in real-time using particle swarm optimization with GPU-accelerated fitness function

“…The tracking times were obtained on LeeWalk sequence [5] as well as on the P1S and P2S image sequences. A software rendering algorithm, which has been used in our previous work [24,34] has been utilized in CPU and CUDA implementations. At the beginning, the rendering algorithm performs the projection and transformation of the model vertices into vertices in image coordinates using the world transformation matrices.…”

“…Each trapezoid of the model undergoes a projection onto 2D image of each camera using parameters of Tsai camera model. The image of the trapezoid is obtained by projecting the corners and then rasterizing the triangles composing the trapezoid [34]. When rendering, back-face culling is executed, which searches for all normals that point away from the viewpoint and skips the associated faces.…”

“…In our system, the CPU is responsible for acquiring images from four calibrated and synchronized cameras as well as processing them. The advantage of such an approach is that the image processing is executed in parallel with the acquisition of the images [34]. The images with the extracted silhouettes and the edges are transferred asynchronously from the host memory to the GPU memory.…”

Performance evaluation of a 3D multi-view-based particle filter for visual object tracking using GPUs and multicore CPUs

“…Rymut and Kwolek [29,30] show a particle swarm optimization implementation on GPU for 3D visual tracking purposes. They get a performance improvement of up to 12.8Â when using the GPU against a single-threaded CPU solution in a high-dimensional articulated tracking problem.…”

Parallel computing on graphics processing units and heterogeneous platforms

“…In , the authors describe how to conduct high‐performance tracking of 3D human motion in real‐time using multi‐view images and particle swarm optimization. The tracking involves configuring the 3D human model, in the pose described by each particle, and then rasterizing it in each particle's 2D plane.…”