Depth Map Refinement for Immersive Video

Mieloch, Dawid; Dziembowski, Adrian; Domański, Marek

doi:10.1109/access.2021.3050554

Cited by 17 publications

(10 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, as depth maps are estimated using compressed views, some degradation of their quality can be seen [15,16]. This compression-induced degradation can be decreased using refinement methods for compression artifacts reduction [17] or depth map enhancement [18,19]. However, the use of additional postprocessing steps increases the valuable time from acquiring the bitstream to present the virtual view to a final viewer.…”

Section: Feature-driven Dsdementioning

confidence: 99%

Recursive block splitting in feature‐driven decoder‐side depth estimation

et al. 2022

Self Cite

View full text Add to dashboard Cite

This paper presents a study on the use of encoder-derived features in decoderside depth estimation. The scheme of multiview video encoding does not require the transmission of depth maps (which carry the geometry of a threedimensional scene) as only a set of input views and their parameters are compressed and packed into the bitstream, with a set of features that could make it easier to estimate geometry in the decoder. The paper proposes novel recursive block splitting for the feature extraction process and evaluates different scenarios of feature-driven decoder-side depth estimation, performed by assessing their influence on the bitrate of metadata, quality of the reconstructed video, and time of depth estimation. As efficient encoding of multiview sequences became one of the main scopes of the video encoding community, the experimental results are based on the "geometry absent" profile from the incoming MPEG Immersive video standard. The results show that the quality of synthesized views using the proposed recursive block splitting outperforms that of the state-of-the-art approach.

show abstract

Section: Feature-driven Dsdementioning

confidence: 99%

Recursive block splitting in feature‐driven decoder‐side depth estimation

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…Furthermore, in addition to the four given quantization parameter pairs (QPT, QPD) for video compression with VVenC/VVdeC [16], another quantization parameter pair is added to show the performance at low bit-rate. The set of (QPT, QPD) pairs is: {(22, 4), (27,7), (32, 11), (37, 15)}. For the sequences generated by the proposed method, we adjusted the QPT quantization parameter due to the spatial resolution downsampling to the following: {16, 21, 26, 31} in order to result to a similar bitrate range as that of the anchors generated with benchmark method.…”

Section: Test Conditionsmentioning

confidence: 99%

“…bad blending and disocclusions) that often degrade the quality of experience [5]. Efforts to improve the performance of TMIV have primarily focused on improving the depth estimation and on the synthesis at the decoder side [6,7]. In [6], a patch-based estimation of the depth at the decoder side is proposed.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A CNN-based Post-Processor for Perceptually-Optimized Immersive Media Compression

Katsenou¹,

Zhang²,

Bull³

2022

Preprint

View full text Add to dashboard Cite

In recent years, resolution adaptation based on deep neural networks has enabled significant performance gains for conventional (2D) video codecs. This paper investigates the effectiveness of spatial resolution resampling in the context of immersive content. The proposed approach reduces the spatial resolution of input multi-view videos before encoding, and reconstructs their original resolution after decoding. During the up-sampling process, an advanced CNN model is used to reduce potential re-sampling, compression and synthesis artefacts. This work has been fully tested with the TMIV coding standard using a Versatile Video Coding (VVC) codec. The results demonstrate that the proposed method achieves significant rate quality performance improvement for the majority of the test sequences, with an average BD-VMAF improvement of 3.07 over all sequences.

show abstract

“…Consequently, it generates output videos defined as atlases. Recent research on MIV has focused on efficient compression methods for immersive videos, accomplishing quality improvement over the existing MIV [11], [12], [13], [14], [15], [16], [17]. Although MIV recommends limiting the number of decoder instances to four, four decoder instances is a challenge for legacy systems that normally contains one hardware decoder.…”

Section: Introductionmentioning

confidence: 99%

VVC Subpicture-Based Frame Packing for MPEG Immersive Video

2022

View full text Add to dashboard Cite

The moving picture experts group (MPEG) video coding group started an immersive video coding standard project to accomplish acceptable bandwidth and decoding resources for multiple immersive videos with texture (color) and geometry (depth) information. The MPEG immersive video (MIV) coding standard provides functionality to remove inter-view redundancy and merge the residuals into videos defined as atlases. Although MIV reduces the number of videos to decode, an efficient decoder instance reduction method has not been investigated. This study proposes a frame packing implementation in MIV software. The proposed method conducts frame packing for geometry atlases to align with texture atlases, encodes them as subpicture bitstreams, and merges one texture / geometry atlas into a single bitstream with a bitstream merger. Furthermore, the proposed method provides control over the number of decoder instances without significantly increasing the computational complexity compared to existing frame packing methods. Therefore, the proposed method was accepted in MIV and will be implemented on the recent version of the reference software of MIV.

show abstract

Depth Map Refinement for Immersive Video

Cited by 17 publications

References 40 publications

Recursive block splitting in feature‐driven decoder‐side depth estimation

Recursive block splitting in feature‐driven decoder‐side depth estimation

A CNN-based Post-Processor for Perceptually-Optimized Immersive Media Compression

VVC Subpicture-Based Frame Packing for MPEG Immersive Video

Contact Info

Product

Resources

About