Adaptive camera-based color mapping for mixed-reality applications

Knecht, Martin W.; Traxler, Christoph; Purgathofer, Werner; Wimmer, Michael

doi:10.1109/ismar.2011.6162884

Cited by 8 publications

(10 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Currently the appearance of virtual object is enhanced using linear photometric compensation (gain and bias). With dense correspondence between the model and the camera image provided by the OF, tone mapping [20] could be employed to better compensate for differences in the photometric spaces of the camera and the model, making the rendered colors more closely match those of the real camera.…”

Section: Discussionmentioning

confidence: 99%

“…We therefore employ a simple linear adjustment using estimated average gain and bias values for all real object pixels to camera pixel correspondences in the scene. However, color space bias could be modeled as a tone mapping based on dynamic camera image histograms, such as with the approach of Knecht et al [20]. We leave this for future work.…”

Section: Corrections To Virtual Object Pixelsmentioning

confidence: 99%

See 1 more Smart Citation

Pixel-wise closed-loop registration in video-based augmented reality

Zheng

Schmalstieg

Welch

2014

2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

View full text Add to dashboard Cite

Comparison between a conventional open-loop registration approach (a) and our closed-loop registration approach employing both world-space pose refinement (b) and screen-space pixel-wise corrections (c). The dragon, the square & axes object and shadows are augmented. In (a), errors in estimates of camera intrinsics and extrinsics (6DOF pose) result in visible misregistration that is neither measured nor corrected as part of a conventional open-loop approach. Such registration errors include direct virtual object misregistration (e.g., the square & axes object), and "phantom" object misregistration errors including incorrect real-to-virtual occlusions (e.g., between the tower and the dragon) and associated shading effects between the real and the virtual (e.g., virtual shadow cast by the tower). See the "zoomed in" portions of the images. In our closed-loop approach, registration errors are detected using a model of the real scene, and corrected in both world space using camera pose refinement (b) and screen space using pixel-wise corrections (c) to address both rigid and non-rigid registration errors. The final result (c) is spatially accurate and exhibits visually coherent registration. ABSTRACTIn Augmented Reality (AR), visible misregistration can be caused by many inherent error sources, such as errors in tracking, calibration, and modeling. In this paper we present a novel pixel-wise closed-loop registration framework that can automatically detect and correct registration errors using a reference model comprised of the real scene model and the desired virtual augmentations. Registration errors are corrected in both global world space via camera pose refinement, and local screen space via pixel-wise corrections, resulting in spatially accurate and visually coherent registration. Specifically we present a registration-enforcing model-based tracking approach that weights important image regions while refining the camera pose estimates (from any conventional tracking method) to achieve better registration, even in the case of modeling errors. To deal with remaining errors, which can be rigid or non-rigid, we compute the optical flow between the camera image and the real model image rendered with the refined pose, enabling direct screenspace pixel-wise corrections to misregistration. The estimated flow field can be applied to improve registration in two distinct ways: (1) forward warping of modeled on-real-object-surface augmentations (e.g., object re-texturing) into the camera image, leading to surface details that are not present in the virtual object; and (2) backward warping of the camera image into the real scene model, preserving the full use of the dense geometry buffer (depth in particular) pro- * e-mail: zhengf@cs.unc.edu † e-mail: schmalstieg@tugraz.at ‡ e-mail: welch@ucf.edu vided by the combined real-virtual model for registration, leading to pixel accurate real-virtual occlusion. We discuss the trade-offs between, and different use cases of, forward and backward warping with model-based tracking in t...

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Corrections To Virtual Object Pixelsmentioning

confidence: 99%

Pixel-wise closed-loop registration in video-based augmented reality

Zheng

Schmalstieg

Welch

2014

2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)

View full text Add to dashboard Cite

show abstract

“…One of the other issues with AR, is the need for exact illumination with respect to the environments to make the system maximally realistic [52] [35] [53] [54] [55]. …”

Section: Related Workmentioning

confidence: 99%

LivePhantom: Retrieving Virtual World Light Data to Real Environments

2016

View full text Add to dashboard Cite

To achieve realistic Augmented Reality (AR), shadows play an important role in creating a 3D impression of a scene. Casting virtual shadows on real and virtual objects is one of the topics of research being conducted in this area. In this paper, we propose a new method for creating complex AR indoor scenes using real time depth detection to exert virtual shadows on virtual and real environments. A Kinect camera was used to produce a depth map for the physical scene mixing into a single real-time transparent tacit surface. Once this is created, the camera’s position can be tracked from the reconstructed 3D scene. Real objects are represented by virtual object phantoms in the AR scene enabling users holding a webcam and a standard Kinect camera to capture and reconstruct environments simultaneously. The tracking capability of the algorithm is shown and the findings are assessed drawing upon qualitative and quantitative methods making comparisons with previous AR phantom generation applications. The results demonstrate the robustness of the technique for realistic indoor rendering in AR systems.

show abstract

“…In order to blend seamlessly the virtual assets with the real world we need to replicate the lighting [6] and color response [15] of the real scene. To achieve that, a unique high dynamic range image (HDRI) of the scene using a 360…”

Section: Automatic Lighting and Gradingmentioning

confidence: 99%

A Game Engine as a Generic Platform for Real-Time Previz-on-Set in Cinema Visual Effects

Goussencourt

Dellac²,

Bertolino

2015

Advanced Concepts for Intelligent Vision Systems

View full text Add to dashboard Cite

Abstract. We present a complete framework designed for film production requiring live (pre) visualization. This framework is based on a famous game engine, Unity R . Actually, game engines possess many advantages that can be directly exploited in real-time pre-vizualization, where real and virtual worlds have to be mixed. In the work presented here, all the steps are performed in Unity: from acquisition to rendering. To perform real-time compositing that takes into account occlusions that occur between real and virtual elements as well as to manage physical interactions of real characters towards virtual elements, we use a low resolution depth map sensor coupled to a high resolution film camera. The goal of our system is to give the film director's creativity a flexible and powerful tool on stage, long before post-production.

show abstract

Adaptive camera-based color mapping for mixed-reality applications

Cited by 8 publications

References 6 publications

Pixel-wise closed-loop registration in video-based augmented reality

Pixel-wise closed-loop registration in video-based augmented reality

LivePhantom: Retrieving Virtual World Light Data to Real Environments

A Game Engine as a Generic Platform for Real-Time Previz-on-Set in Cinema Visual Effects

Contact Info

Product

Resources

About