Unified multi-lateral filter for real-time depth map enhancement

Garcia, Frederic; Aouada, Djamila; Mirbach, Bruno; Solignac, Thomas; Ottersten, Björn

doi:10.1016/j.imavis.2015.06.008

Cited by 19 publications

(14 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In multi-modal fusion approaches, the HR 2D image is a guidance image used to enhance the resolution of the LR depth image. We consider three representative algorithms; the Pixel Weighted Average Strategy (PWAS) filter [25], and its improved version called Unified Multi-Lateral (UML) filter [26] which are two fusion filters based on the concept of bilateral filtering, and the method of umpsampling using anisotropic Total Generalized Variation (aTGV) which is based on a global optimization. The results for one frame are given in Fig.…”

Section: Comparison With State-of-art Methodsmentioning

confidence: 99%

“…This category of methods is based on the assumption that there is a correspondence between the edges of a depth map and the edges of another modality of a better quality, often chosen to be a 2D intensity image of the same scene [19], [20], [21], [22], [23], [24], [26]. Such an image is considered to be a guidance image whose properties, in terms of structure, signal to noise ratio (SNR) and resolution, can be transferred to its corresponding depth map in order to obtain an enhanced version.…”

Section: Multi-modal Fusion Approachesmentioning

confidence: 99%

“…In 2007, Kopf et al [20] and Yang et al [21] proposed the concept of joint bilateral upsampling (JBU), which is an adapted version of the bilateral filter [27] for fusing a low resolution (LR) depth map with a high resolution (HR) 2D image. In [22], [26], the JBU multi-modal filter was extended to considering not only a 2D image as a guidance image but also the depth map itself. These weighting-based filters may be viewed as implicit guided regularization approaches.…”

Section: Multi-modal Fusion Approachesmentioning

confidence: 99%

See 2 more Smart Citations

Real-Time Enhancement of Dynamic Depth Videos with Non-Rigid Deformations

Ismaeil

Aouada

Solignac

et al. 2017

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

Abstract-We propose a novel approach for enhancing depth videos containing non-rigidly deforming objects. Depth sensors are capable of capturing depth maps in real-time but suffer from high noise levels and low spatial resolutions. While solutions for reconstructing 3D details in static scenes, or scenes with rigid global motions have been recently proposed, handling unconstrained non-rigid deformations in relative complex scenes remains a challenge. Our solution consists in a recursive dynamic multi-frame superresolution algorithm where the relative local 3D motions between consecutive frames are directly accounted for. We rely on the assumption that these 3D motions can be decoupled into lateral motions and radial displacements. This allows to perform a simple local per-pixel tracking where both depth measurements and deformations are dynamically optimized. The geometric smoothness is subsequently added using a multi-level L 1 minimization with a bilateral total variation regularization. The performance of this method is thoroughly evaluated on both real and synthetic data. As compared to alternative approaches, the results show a clear improvement in reconstruction accuracy and in robustness to noise, to relative large non-rigid deformations, and to topological changes. Moreover, the proposed approach, implemented on a CPU, is shown to be computationally efficient and working in real-time.

show abstract

Section: Comparison With State-of-art Methodsmentioning

confidence: 99%

Section: Multi-modal Fusion Approachesmentioning

confidence: 99%

Section: Multi-modal Fusion Approachesmentioning

confidence: 99%

See 1 more Smart Citation

Real-Time Enhancement of Dynamic Depth Videos with Non-Rigid Deformations

Ismaeil

Aouada

Solignac

et al. 2017

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, they are computationally intensive and thus not suitable in real-time system. Joint bilateral filter and its variations [14,15] are often used to increase the spatial resolution. Garica et al, [14] proposed a novel unified multi-lateral filter.…”

Section: Related Workmentioning

confidence: 99%

“…Joint bilateral filter and its variations [14,15] are often used to increase the spatial resolution. Garica et al, [14] proposed a novel unified multi-lateral filter. It can generate super resolution depth map and suppress the unwanted artifacts.…”

Section: Related Workmentioning

confidence: 99%