Virtual Reality (VR) contents comprise 360°×180° seamless panoramic videos, stitched from multiple overlapping video streams. A recent increase in demand of VR content has resulted in the supply of commercially available solutions which, though cheap, lack scalability and quality. In this paper, we propose an end-to-end VR system for stitching full spherical contents. The VR system is composed of camera rig calibration and stitching modules. The calibration module performs geometric alignment of camera rig. The stitching module transforms texture from camera or video stream into VR stream using lookup tables (LUTs) and blend masks (BMs). In this work, our main contribution is improvement of stitching quality. First, we propose a feature preprocessing method that filters out inconsistent, error-prone features. Secondly, we propose a geometric alignment method that outperforms state-of-the-art VR stitching solutions. We tested our system on diverse image sets and obtained state-of-the-art geometric alignment. Moreover, we achieved real-time stitching of camera and video streams up to 120 fps at 4K resolution. After stitching, we encode VR content for IP multicasting.

This paper presents a keypoint detection method based on the Laplacian of Gaussian (LoG). In contrast to the Difference of Gaussian (DoG)-based keypoint detection method used in Scale Invariant Feature Transform (SIFT), we focus on the LoG operator and its higher order derivatives. We provide mathematical analogies between higher order DoG (HDoG) and higher order LoG (HLoG) and experimental results to show the effectiveness of the proposed HLoG-based keypoint detection method. The performance of the HLoG is evaluated with four different tests: i) a repeatability test of the keypoints detected across images under various transformations, ii) image retrieval, iii) panorama stitching and iv) 3D reconstruction. The proposed HLoG method provides comparable performance to HDoG and the combination of HLoG and HDoG provides significant improvements in various keypoint-related computer vision problems.

No abstract

The moving picture experts group (MPEG) immersive video (MIV) technology has been actively developed and standardized to efficiently deliver immersive video to viewers in order for them to experience immersion and realism in various realistic and virtual environments. Such services are provided by MIV technology, which uses multiview videos as input. The pruning process, which is an important component of MIV technology, reduces interview redundancy in multiviews videos. The primary aim of the pruning process is to reduce the amount of data that available video codec must handle. In this study, two approaches are presented to improve the existing pruning algorithm. The first method determines the order in which images are pruned. The amount of overlapping region between the source views is then used to determine the pruning order. The second method considers global region‐wise color similarity to minimize matching ambiguity when determining the pruning area. The proposed methods are evaluated under common test condition of MIV, and the results show that incorporating the proposed methods can improve both objective and subjective quality.

No abstract