Interactive Augmented Omnidirectional Video with Realistic Lighting

Michiels, Nick; Jorissen, Lode; Put, Jeroen; Bekaert, Philippe

doi:10.1007/978-3-319-13969-2_19

Cited by 9 publications

(7 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most of them use a method of applying camera tracking to perspective views of a 360 • video clip before the stitching process. One such method proposed by Michiels et al uses a perspective view from one of the 360 • camera rigs to obtain an undistorted image for eliminating the stitching errors [13]. In addition, Huang et al proposed a method for obtaining stable tracking results, which uses an image correction by overlapping the point where the distortion occurs with the position difference between frames [14].…”

Section: Stereo Visionmentioning

confidence: 99%

A Novel Real-Time Virtual 3D Object Composition Method for 360° Video

Lee

Gentet

et al. 2020

Applied Sciences

View full text Add to dashboard Cite

As highly immersive virtual reality (VR) content, 360° video allows users to observe all viewpoints within the desired direction from the position where the video is recorded. In 360° video content, virtual objects are inserted into recorded real scenes to provide a higher sense of immersion. These techniques are called 3D composition. For a realistic 3D composition in a 360° video, it is important to obtain the internal (focal length) and external (position and rotation) parameters from a 360° camera. Traditional methods estimate the trajectory of a camera by extracting the feature point from the recorded video. However, incorrect results may occur owing to stitching errors from a 360° camera attached to several high-resolution cameras for the stitching process, and a large amount of time is spent on feature tracking owing to the high-resolution of the video. We propose a new method for pre-visualization and 3D composition that overcomes the limitations of existing methods. This system achieves real-time position tracking of the attached camera using a ZED camera and a stereo-vision sensor, and real-time stabilization using a Kalman filter. The proposed system shows high time efficiency and accurate 3D composition.

show abstract

Section: Stereo Visionmentioning

confidence: 99%

A Novel Real-Time Virtual 3D Object Composition Method for 360° Video

Lee

Gentet

et al. 2020

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Iorns et al [22] developed a system to use a live streaming 360 • video as an input for image-based lighting where real-time shadowing and reflection were investigated. Michiels et al [23] used an appropriate 360 • environment map linked to a car position for real-time lighting added to the rendering equation.…”

Section: Related Workmentioning

confidence: 99%

An Empirical Evaluation of the Performance of Real-Time Illumination Approaches: Realistic Scenes in Augmented Reality

Alhakamy

Tüceryan

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Augmented, Virtual, and Mixed Reality (AR/VR/MR) systems have been developed in general, with many of these applications having accomplished significant results, rendering a virtual object in the appropriate illumination model of the real environment is still under investigation. The entertainment industry has presented an astounding outcome in several media form, albeit the rendering process has mostly been done offline. The physical scene contains the illumination information which can be sampled and then used to render the virtual objects in real-time for realistic scene. In this paper, we evaluate the accuracy of our previous and current developed systems that provide real-time dynamic illumination for coherent interactive augmented reality based on the virtual object's appearance in association with the real world and related criteria. The system achieves that through three simultaneous aspects. (1) The first is to estimate the incident light angle in the real environment using a live-feed 360 • camera instrumented on an AR device. (2) The second is to simulate the reflected light using two routes: (a) global cube map construction and (b) local sampling. (3) The third is to define the shading properties for the virtual object to depict the correct lighting assets and suitable shadowing imitation. Finally, the performance efficiency is examined in both routes of the system to reduce the general cost. Also, The results are evaluated through shadow observation and user study.

show abstract

“…Recently Michiels et al [34] demonstrate the usage of captured 360°video to render virtual objects into a reconstructed scene. Their panoramic video capturing hardware is fixed to a vehicle which is driven through a real life environment.…”

Section: Real-time Ibl Using Omnidirectional Videomentioning

confidence: 99%

“…There are however still several major drawbacks to this technique. The SRBF Figure 3.3: Results from [34]. Lack of HDR environment maps leads to inaccurate and thus somewhat unconvincing lighting, as can be seen with the lower right sphere appearing darker than the surroundings, and the upper right car appearing more blue than would be expected from looking at the environment.…”

Section: Real-time Ibl Using Omnidirectional Videomentioning

confidence: 99%

Real-Time Image Based Lighting for 360-Degree Panoramic Video

Iorns

Rhee

2016

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The application of the newly popular content medium of 360 degree panoramic video to the widely used offline lighting technique of image based lighting is explored, and a system solution for real-time image based lighting of virtual objects using only the provided 360 degree video for lighting is developed. The system solution is suitable for use on live streaming video input, and is shown to run on consumer grade graphics hardware at the high resolutions and framerates necessary for comfortable viewing on head mounted displays, rendering at over 60 frames per second for stereo output at 1182x1464 per eye on a mid-range graphics card. Its use in several real-world applications is also studied, and extension to consider real-time shadowing and reflection is explored.

show abstract

Interactive Augmented Omnidirectional Video with Realistic Lighting

Cited by 9 publications

References 28 publications

A Novel Real-Time Virtual 3D Object Composition Method for 360° Video

A Novel Real-Time Virtual 3D Object Composition Method for 360° Video

An Empirical Evaluation of the Performance of Real-Time Illumination Approaches: Realistic Scenes in Augmented Reality

Real-Time Image Based Lighting for 360-Degree Panoramic Video

Contact Info

Product

Resources

About