Patch Decoder-Side Depth Estimation In Mpeg Immersive Video

Milovanović, Marta; Henry, Félix; Cagnazzo, Marco; Jung, Joël

doi:10.1109/icassp39728.2021.9414056

Cited by 8 publications

(5 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, in a situation where patch atlases are used in the DSDE context [28], a depth estimator may have to deal with small patches and significantly less information. Given that such an approach can be quite complex and decreases the number of applicable depth estimation methods, the current DSDE framework is only utilizing full frames.…”

Section: A Requirements For a Decoder-side Depth Estimatormentioning

confidence: 99%

“…Since sufficient evidence has been presented to prove the relevance of this DSDE approach in immersive video coding [2], [4], [28], one of the MIV profiles implements this principle. The MIV Decoder-Side Depth Estimation (DSDE) using Geometry Absent profile allows MIV bitstreams to not include depth maps, keeping only the information required to reconstruct it.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Overview and Efficiency of Decoder-Side Depth Estimation in MPEG Immersive Video

Mieloch¹,

Garus

Milovanović

et al. 2022

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

View full text Add to dashboard Cite

This paper presents the overview and rationale behind the Decoder-Side Depth Estimation (DSDE) mode of the MPEG Immersive Video (MIV) standard, using the Geometry Absent profile, for efficient compression of immersive multiview video. A MIV bitstream generated by an encoder operating in the DSDE mode does not include depth maps. It only contains the information required to reconstruct them in the client or in the cloud: decoded views and metadata. The paper explains the technical details and techniques supported by this novel MIV DSDE mode. The description additionally includes the specification on Geometry Assistance Supplemental Enhancement Information which helps to reduce the complexity of depth estimation, when performed in the cloud or at the decoder side. The depth estimation in MIV is a non-normative part of the decoding process, therefore, any method can be used to compute the depth maps. This paper lists a set of requirements for depth estimation, induced by the specific characteristics of the DSDE. The depth estimation reference software, continuously and collaboratively developed with MIV to meet these requirements, is presented in this paper. Several original experimental results are presented. The efficiency of the DSDE is compared to two MIV profiles. The combined non-transmission of depth maps and efficient coding of textures enabled by the DSDE leads to efficient compression and rendering quality improvement compared to the usual encoder-side depth estimation. Moreover, results of the first evaluation of state-of-the-art multiview depth estimators in the DSDE context, including machine learning techniques, are presented.

show abstract

Section: A Requirements For a Decoder-side Depth Estimatormentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Overview and Efficiency of Decoder-Side Depth Estimation in MPEG Immersive Video

Mieloch¹,

Garus

Milovanović

et al. 2022

IEEE Trans. Circuits Syst. Video Technol.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Such features may be used for increasing the quality of depth maps estimated at the decoder side, decreasing the total decoding time [19], or for efficient encoding of multi-plane images (MPI) [20]. Other interesting approaches were presented in [21], where a possibility of utilizing motion compensation for DSDE was examined, and [22] -where depth estimation is performed for transmitted full views and smaller patches transmitted within texture atlases [2].…”

Section: Related Workmentioning

confidence: 99%

Advanced HEVC Screen Content Coding for MPEG Immersive Video

2022

View full text Add to dashboard Cite

This paper presents the modified HEVC Screen Content Coding (SCC) that was adapted to be more efficient as an internal video coding of the MPEG Immersive Video (MIV) codec. The basic, unmodified SCC is already known to be useful in such an application. However, in this paper, we propose three additional improvements to SCC to increase the efficiency of immersive video coding. First, we analyze using the quarter-pel accuracy in the intra block copy technique to provide a more effective search of the best candidate block to be copied in the encoding process. The second proposal is the use of tiles to allow inter-view prediction inside MIV atlases. The last proposed improvement is the addition of the MIV bitstream parser in the HEVC encoder that enables selecting the most efficient coding configuration depending on the type of currently encoded data. The experimental results show that the proposal increases the compression efficiency for natural content sequences by almost 7% and simultaneously decreases the computational time of encoding by more than 15%, making the proposal very valuable for further research on immersive video coding.

show abstract

“…bad blending and disocclusions) that often degrade the quality of experience [5]. Efforts to improve the performance of TMIV have primarily focused on improving the depth estimation and on the synthesis at the decoder side [6,7]. In [6], a patch-based estimation of the depth at the decoder side is proposed.…”

Section: Introductionmentioning

confidence: 99%

“…Efforts to improve the performance of TMIV have primarily focused on improving the depth estimation and on the synthesis at the decoder side [6,7]. In [6], a patch-based estimation of the depth at the decoder side is proposed. In [7], an iterative depth refinement process is presented that improves inter-view consistency.…”

Section: Introductionmentioning

confidence: 99%

A CNN-based Post-Processor for Perceptually-Optimized Immersive Media Compression

Katsenou¹,

Zhang²,

Bull³

2022

Preprint

View full text Add to dashboard Cite

In recent years, resolution adaptation based on deep neural networks has enabled significant performance gains for conventional (2D) video codecs. This paper investigates the effectiveness of spatial resolution resampling in the context of immersive content. The proposed approach reduces the spatial resolution of input multi-view videos before encoding, and reconstructs their original resolution after decoding. During the up-sampling process, an advanced CNN model is used to reduce potential re-sampling, compression and synthesis artefacts. This work has been fully tested with the TMIV coding standard using a Versatile Video Coding (VVC) codec. The results demonstrate that the proposed method achieves significant rate quality performance improvement for the majority of the test sequences, with an average BD-VMAF improvement of 3.07 over all sequences.

show abstract

Patch Decoder-Side Depth Estimation In Mpeg Immersive Video

Cited by 8 publications

References 9 publications

Overview and Efficiency of Decoder-Side Depth Estimation in MPEG Immersive Video

Overview and Efficiency of Decoder-Side Depth Estimation in MPEG Immersive Video

Advanced HEVC Screen Content Coding for MPEG Immersive Video

A CNN-based Post-Processor for Perceptually-Optimized Immersive Media Compression

Contact Info

Product

Resources

About