A High-Quality VR Calibration and Real-Time Stitching Framework Using Preprocessed Features

Abstract

Virtual Reality (VR) contents comprise 360°×180° seamless panoramic videos, stitched from multiple overlapping video streams. A recent increase in demand of VR content has resulted in the supply of commercially available solutions which, though cheap, lack scalability and quality. In this paper, we propose an end-to-end VR system for stitching full spherical contents. The VR system is composed of camera rig calibration and stitching modules. The calibration module performs geometric alignment of camera rig. The stitching module transforms texture from camera or video stream into VR stream using lookup tables (LUTs) and blend masks (BMs). In this work, our main contribution is improvement of stitching quality. First, we propose a feature preprocessing method that filters out inconsistent, error-prone features. Secondly, we propose a geometric alignment method that outperforms state-of-the-art VR stitching solutions. We tested our system on diverse image sets and obtained state-of-the-art geometric alignment. Moreover, we achieved real-time stitching of camera and video streams up to 120 fps at 4K resolution. After stitching, we encode VR content for IP multicasting.

A study on presence quality and cybersickness in 2D, smartphone, and VR

“…PTGui [38] and Hugin [39]) and provided as supplementary materials. The quality improvements in our framework result in the higher presence and lower cybersickness as compared to the commercial solutions in connection with the improvements in the geometric alignment reported in [40].…”

“…presence and cybersickness) is measured in experimental setups similar to previous studies [20][21][22][23][24][25][26][27]. Second, a comparison of the presence and cybersickness ratings of participants towards VR contents created from our VR framework [40] and commercial solutions [38,39] is performed. Third, the VR QoE is compared among conventional 2D displays and VR headset in a setup similar to [16,51].…”

“…In this section, firstly, we will discuss our VR framework [40] for capturing and rendering high-quality contents. Secondly, we will illustrate the experimental setup for processing presence quality and cybersickness feedback from participants.…”

“…The experimental setup uses a VR framework [40] with four main components that are based on traditional panorama stitching and rendering techniques [41][42][43][44][45][46][47][48][49]. The first component is the camera rig, the second component is the camera calibration module, the third component is the real-time stitching module and the fourth one consists of VR content consumption devices.…”

“…The results show that the improvements in VR quality improve the presence and reduce cybersickness irrespective of the type of display. The same datasets are used to create VR contents with our VR framework [40] and commercial solutions (i.e. PTGui [38] and Hugin [39]) and provided as supplementary materials.…”