Standardization Status of Immersive Video Coding

Abstract

Based on increasing availability of capture and display devices dedicated to immersive media, coding, and transmission of these media has recently become a highest-priority subject of standardization. Different levels of immersiveness are defined with respect to an increasing degree of freedom in terms of movements of the observer within the immersive media scene. The level ranges from three degrees of freedom allowing the user to look around in all directions from a fixed point of view to six degrees of freedom, where the user can freely alter the viewpoint within the immersive media scene. The moving pictures experts group (MPEG) of ISO/IEC is developing a standards suite on "Coded Representation of Immersive Media," called MPEG-I, to provide technical solutions for building blocks of the media transmission chain, ranging from architecture, systems tools, coding of video and audio signals, to point clouds and timed text. In this paper, an overview on recent and ongoing standardization efforts in this area is presented. While some specifications, such as high efficiency video coding or version 1 of the omnidirectional media format, are already available, other activities are under development or in the exploration phase. This paper addresses the status of these efforts with a focus on video signals, indicates the development timelines, summarizes the main technical details, and provides pointers to further points of reference.

Bypassing Depth Maps Transmission For Immersive Video Coding

“…A work item referred to as ISO/IEC 23090 and known as MPEG-I has been started in 2017. Current state-of-the-art technologies for immersive video coding can be categorized according to the degrees of freedom (DoF) of positioning and orientation of the viewer [1], [2]. Enabling merely yaw, pitch and roll movements of the users head provides the viewer a total of 3DoF.…”

Multichannel Blind Source Separation Based on Evanescent-Region-Aware Non-Negative Tensor Factorization in Spherical Harmonic Domain

“…The number of iterations was set to 100 to guarantee the convergence of all the methods. 2 https://mhacoustics.com/ Fig. 4: Evanescent-region-aware mask generated with threshold value n = ekR mic /2 where R mic is set to 0.042 m. Fig.…”

“…The METU SPARG Eigenmike em32 Acoustic Impulse Response Dataset v1.0 [48] containing a large set of impulse responses recorded by the em32 Eigenmike was used. The room where the recording was conducted had a reverberation of T 60 = 1.12 s, and its shape was rectangular (6.5 × 8.3 × 2.9 m) (see [2] for a detailed description of the dataset). Impulse responses from various source positions were chosen randomly from 240 candidates.…”

“…It consists of pairs of object metadata and corresponding waveform signals. Placing an audio object anywhere in space gives listeners an immersive experience that can serve as the audio part of the six degrees of freedom (6DOF) system proposed in MPEG-I [2]. Although the HOA format can be rotated with respect to the origin in the spherical coordinate system, and many authors have made considerable effort to reduce the computational cost to make the system feasible in the three degree of freedom (3DOF) system [3]- [5], the format cannot be easily adapted to the 6DOF framework where translations along the xyz axes are required.…”

Joint effects of depth-aiding augmentations and viewing positions on the quality of experience in augmented telepresence

“…2. This setup provided stereoscopic AR that reacts to omnidirectional 6-Degrees of Freedom (DoF) headset movement, as per DoF definitions in [78]. Since the camera positions were The augmentation for each eye is anchored in the 3D virtual space on a line connecting the virtual eye position to a texture point on the sphere surface.…”