Dynamic super resolution of depth sequences with non-rigid motions

Ismaeil, Kassem Al; Aouada, Djamila; Mirbach, Bruno; Ottersten, Björn

doi:10.1109/icip.2013.6738136

Cited by 9 publications

(21 citation statements)

References 21 publications

(28 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To that end, we use the same moving chairs data presented in [8] and in [10]. The used setup to capture this data was an LR ToF camera, the 3D MLI of resolution (56 Â 61) [65], mounted in the ceiling at a height of 2.5 m, and coupled with an HR 2D camera, the Dragonfly2 CCD camera of resolution (648 Â 488) from Point Grey.…”

Section: Comparison With State-of-art Methodsmentioning

confidence: 99%

“…Then we downsample the obtained depth sequence with r ¼ 4 and further degrade it with additive Gaussian noise with standard deviation s varying from 0 to 50 mm. The created LR noisy depth sequence is then super-resolved using state-of-art methods: the conventional bicubic interpolation, UP-SR [8], SISR [31], and the proposed recUP-SR. Table 1 reports the 3D reconstruction error of each method at different noise levels. Then, we compare the accuracy of the reconstructed 3D super-resolved scene with state-of-art results.…”

Section: Comparison With State-of-art Methodsmentioning

confidence: 99%

“…Indeed, it exploits the deformations collected over time in an inverse SR reconstruction framework. UP-SR was shown in [8] and [10] to have a higher reconstruction accuracy as compared to representative methods from the first and second categories of depth enhancement methods. Its major drawbacks, however, are its limitation to lateral motions and its computational complexity.…”

Section: Dynamic Multi-frame Approachesmentioning

confidence: 99%

“…In [8], [9], [10], we proposed the UP-SR algorithm, which stands for Upsampling for Precise Super-Resolution, as the first dynamic multi-frame depth video super-resolution (SR) algorithm that can enhance depth videos containing non-rigidly deforming scenes without any prior assumption on the number of moving objects they contain or on the topology of these objects. These advantages were possible thanks to a direct processing on depth maps without using connectivity information inherent to meshing as used in subsequent methods, namely, KinectDeform [11] and DynamicFusion [12].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Real-Time Enhancement of Dynamic Depth Videos with Non-Rigid Deformations

Ismaeil

Aouada

Solignac

et al. 2017

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

Abstract-We propose a novel approach for enhancing depth videos containing non-rigidly deforming objects. Depth sensors are capable of capturing depth maps in real-time but suffer from high noise levels and low spatial resolutions. While solutions for reconstructing 3D details in static scenes, or scenes with rigid global motions have been recently proposed, handling unconstrained non-rigid deformations in relative complex scenes remains a challenge. Our solution consists in a recursive dynamic multi-frame superresolution algorithm where the relative local 3D motions between consecutive frames are directly accounted for. We rely on the assumption that these 3D motions can be decoupled into lateral motions and radial displacements. This allows to perform a simple local per-pixel tracking where both depth measurements and deformations are dynamically optimized. The geometric smoothness is subsequently added using a multi-level L 1 minimization with a bilateral total variation regularization. The performance of this method is thoroughly evaluated on both real and synthetic data. As compared to alternative approaches, the results show a clear improvement in reconstruction accuracy and in robustness to noise, to relative large non-rigid deformations, and to topological changes. Moreover, the proposed approach, implemented on a CPU, is shown to be computationally efficient and working in real-time.

show abstract

Section: Comparison With State-of-art Methodsmentioning

confidence: 99%

Section: Comparison With State-of-art Methodsmentioning

confidence: 99%

Section: Dynamic Multi-frame Approachesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Real-Time Enhancement of Dynamic Depth Videos with Non-Rigid Deformations

Ismaeil

Aouada

Solignac

et al. 2017

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

show abstract

“…The proposed solution can handle scenes containing one or more moving objects even non-rigidly without prior assumptions on their shape, and without training. Our algorithm referred to as Upsampling for Precise Super Resolution (UP-SR) builds on our work in [13,14,15]. We herein give a unified framework and provide additional details and proofs, and a more extensive experimental part, where we evaluate the accuracy of the proposed algorithm theoretically and experimentally as function of the SR factor, and the level of contaminations with noise.…”

Section: Introductionmentioning

confidence: 99%

Enhancement of dynamic depth scenes by upsampling for precise super-resolution (UP-SR)

Ismaeil

Aouada

Mirbach

et al. 2016

Computer Vision and Image Understanding

Self Cite

View full text Add to dashboard Cite

Multi-frame super-resolution is the process of recovering a high resolution image or video from a set of captured low resolution images. Super-resolution approaches have been largely explored in 2-D imaging. However, their extension to depth videos is not straightforward due to the textureless nature of depth data, and to their high frequency contents coupled with fast motion artifacts. Recently, few attempts have been introduced where only the super-resolution of static depth scenes has been addressed. In this work, we propose to enhance the resolution of dynamic depth videos with non-rigidly moving objects. The proposed approach is based on a new data model that uses densely upsampled, and cumulatively registered versions of the observed low resolution depth frames. We show the impact of upsampling in increasing the sub-pixel accuracy and reducing the rounding error of the motion vectors. Furthermore, with the proposed cumulative motion estimation, a high registration accuracy is achieved between non-successive upsampled frames with relative large motions. A statistical performance analysis is derived in terms of mean square error explaining the effect of the number of observed frames and the effect of the super-resolution factor at a given noise level. We evaluate the accuracy of the proposed algorithm theoretically and experimentally as function of the SR factor, and the level of contaminations with noise. Experimental results on both real and synthetic data show the effectiveness of the proposed algorithm on dynamic depth videos as compared to state-of-art methods.

show abstract