Deep spherical harmonics light probe estimator for mixed reality games

Marques, Bruno Augusto Dorta; Clua, Esteban; Vasconcelos, Cristina Nader

doi:10.1016/j.cag.2018.09.003

Cited by 15 publications

(8 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To identify lighting conditions of the real scene, such as the position of the light source, many works, Knecht et al (2011);Grosch et al (2007); Kan and Kaufmann (2013); Franke (2013); Kan and Kaufmann (2012); Rohmer et al (2014) and Gierlinger et al (2010), use cameras with fish-eye lens. In Schwandt and Broll (2016) and Pereira et al (2020) however, the position of the light source was previously known, and in Marques et al (2018) A perceived trend was the use of Differential Rendering by Debevec (2008) as in Kan and Kaufmann (2013); Rohmer et al (2017); Franke (2013); Kan and Kaufmann (2012); Gruber et al (2015) and Gruber et al (2012). In Knecht et al (2011) and Knecht et al (2012) a specific version, the Differential Instant Radiosity was used (Knecht et al, 2010).…”

Section: Analysis Of Resultsmentioning

confidence: 99%

“…Part of the works emphasized Augmented Reality systems. The Marques et al (2018) work explores an image-based and hardware-based approach to improve photorealism for rendering synthetic objects in augmented reality. Knecht et al (2012) presented a distributed illumination approach for AR with consistent illumination of virtual objects with direct light, indirect light, and shadows of primary and strong secondary lights.…”

Section: The Selected Papers Proposalsmentioning

confidence: 99%

See 1 more Smart Citation

Photorealism in Mixed Reality: A Systematic Literature Review

Pereira

Júnior

Moreno

2021

IJVR

View full text Add to dashboard Cite

In Augmented Reality systems, virtual objects are combined with real objects, both three dimensional, interactively and at run-time. In an ideal scenario, the user has the feeling that real and virtual objects coexist in the same space and is unable to differentiate the types of objects from each other. To achieve this goal, research on rendering techniques have been conducted in recent years. In this paper, we present a Systematic Literature Review aiming to identify the main characteristics concerning photorealism in Mixed and Augmented Reality systems to find research opportunities that can be further exploited or optimized. The objective is to verify if exists a definition of photorealism in Mixed and Augmented Reality. We present a theoreticalfundamental over the most used methods concerning realism in Computer Graphics. Also, we want to identify the most used methods and tools to enable photorealism in Mixed and Augmented Reality systems.

show abstract

Section: Analysis Of Resultsmentioning

confidence: 99%

Section: The Selected Papers Proposalsmentioning

confidence: 99%

Photorealism in Mixed Reality: A Systematic Literature Review

Pereira

Júnior

Moreno

2021

IJVR

View full text Add to dashboard Cite

show abstract

“…Ref. [51] used the spherical harmonics light probe estimator for light estimation based on deep learning. This method processes a raw image without the need for previous knowledge about the scene.…”

Section: Visualization and Renderingmentioning

confidence: 99%

“…In the chroma key, the use of statically-based techniques for temporal matting and Holoing artifacts is needed [50]. Improvement of the light and shadow models requires temporal and dynamic effects modeling (such as abrupt changes in lighting, dynamic scene, and local light source) in addition to specifying classes of different algorithms for different light paths [51,54]. The LOD model requires study of the appropriate architecture to determine the volume of the process in the server and client [55].…”

Section: Visualization and Renderingmentioning

confidence: 99%

A Review on Mixed Reality: Current Trends, Challenges and Prospects

2020

View full text Add to dashboard Cite

Currently, new technologies have enabled the design of smart applications that are used as decision-making tools in the problems of daily life. The key issue in designing such an application is the increasing level of user interaction. Mixed reality (MR) is an emerging technology that deals with maximum user interaction in the real world compared to other similar technologies. Developing an MR application is complicated, and depends on the different components that have been addressed in previous literature. In addition to the extraction of such components, a comprehensive study that presents a generic framework comprising all components required to develop MR applications needs to be performed. This review studies intensive research to obtain a comprehensive framework for MR applications. The suggested framework comprises five layers: the first layer considers system components; the second and third layers focus on architectural issues for component integration; the fourth layer is the application layer that executes the architecture; and the fifth layer is the user interface layer that enables user interaction. The merits of this study are as follows: this review can act as a proper resource for MR basic concepts, and it introduces MR development steps and analytical models, a simulation toolkit, system types, and architecture types, in addition to practical issues for stakeholders such as considering MR different domains.

show abstract

“…Marques et al [MCV18] address the problem of realistic virtual scene and hand relighting in mixed reality games. The proposed approach assumes that the user's hands are visible in the scene and convey enough information from the surrounding lighting.…”

Section: Human Handsmentioning

confidence: 99%

Deep Neural Models for Illumination Estimation and Relighting: A Survey

Einabadi

Guillemaut

Hilton

2021

Computer Graphics Forum

View full text Add to dashboard Cite

Scene relighting and estimating illumination of a real scene for insertion of virtual objects in a mixed-reality scenario are well-studied challenges in the computer vision and graphics fields. Classical inverse rendering approaches aim to decompose a scene into its orthogonal constituting elements, namely scene geometry, illumination and surface materials, which can later be used for augmented reality or to render new images under novel lighting or viewpoints. Recently, the application of deep neural computing to illumination estimation, relighting and inverse rendering has shown promising results. This contribution aims to bring together in a coherent manner current advances in this conjunction. We examine in detail the attributes of the proposed approaches, presented in three categories: scene illumination estimation, relighting with reflectance-aware scenespecific representations and finally relighting as image-to-image transformations. Each category is concluded with a discussion on the main characteristics of the current methods and possible future trends. We also provide an overview of current publicly available datasets for neural lighting applications.

show abstract

Deep spherical harmonics light probe estimator for mixed reality games

Cited by 15 publications

References 10 publications

Photorealism in Mixed Reality: A Systematic Literature Review

Photorealism in Mixed Reality: A Systematic Literature Review

A Review on Mixed Reality: Current Trends, Challenges and Prospects

Deep Neural Models for Illumination Estimation and Relighting: A Survey

Contact Info

Product

Resources

About