Gaussian material synthesis

Zsolnai-Fehér, Károly; Wonka, Peter; Wimmer, Michael

doi:10.1145/3197517.3201307

Cited by 29 publications

(25 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inspired by the recent success of machine learning for MC denoising [KBS15, BVM∗ 17, CKS∗ 17, VRM∗ 18], light‐field interpolation [KWR16], and for rendering in general [RWG∗ 13, KMM∗ 17, ZWW18], we propose to use a deep network for estimating the sampling PDF from our sparse sets of samples. Deep networks are capable of representing complex, non‐linear relationships, and yet perform well with sparse data like we have in our initial samples.…”

Section: Offline Deep Importance Samplingmentioning

confidence: 99%

Offline Deep Importance Sampling for Monte Carlo Path Tracing

Bako

Meyer²,

DeRose³

et al. 2019

Computer Graphics Forum

View full text Add to dashboard Cite

Although modern path tracers are successfully being applied to many rendering applications, there is considerable interest to push them towards ever‐decreasing sampling rates. As the sampling rate is substantially reduced, however, even Monte Carlo (MC) denoisers–which have been very successful at removing large amounts of noise–typically do not produce acceptable final results. As an orthogonal approach to this, we believe that good importance sampling of paths is critical for producing better‐converged, path‐traced images at low sample counts that can then, for example, be more effectively denoised. However, most recent importance‐sampling techniques for guiding path tracing (an area known as “path guiding”) involve expensive online (per‐scene) training and offer benefits only at high sample counts. In this paper, we propose an offline, scene‐independent deep‐learning approach that can importance sample first‐bounce light paths for general scenes without the need of the costly online training, and can start guiding path sampling with as little as 1 sample per pixel. Instead of learning to “overfit” to the sampling distribution of a specific scene like most previous work, our data‐driven approach is trained a priori on a set of training scenes on how to use a local neighborhood of samples with additional feature information to reconstruct the full incident radiance at a point in the scene, which enables first‐bounce importance sampling for new test scenes. Our solution is easy to integrate into existing rendering pipelines without the need for retraining, as we demonstrate by incorporating it into both the Blender/Cycles and Mitsuba path tracers. Finally, we show how our offline, deep importance sampler (ODIS) increases convergence at low sample counts and improves the results of an off‐the‐shelf denoiser relative to other state‐of‐the‐art sampling techniques.

show abstract

Section: Offline Deep Importance Samplingmentioning

confidence: 99%

Offline Deep Importance Sampling for Monte Carlo Path Tracing

Bako

Meyer²,

DeRose³

et al. 2019

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…Maximov et al [MRF18] introduced the concept of "deep appearance maps", which use a small fully connected network as a material descriptor. Zsolnai-Fehér et al [ZFWW18] use a neural network to render previews of materials with static scene geometry. The inverse problem has also been the focus of recent works [LDPT17, LSC18, DAD * 18] that take an image as input and output estimated BRDF (or SVBRDF) parameters.…”

Section: Statisticalmentioning

confidence: 99%

Neural BTF Compression and Interpolation

Rainer

Jakob

Ghosh

et al. 2019

Computer Graphics Forum

View full text Add to dashboard Cite

The Bidirectional Texture Function (BTF) is a data‐driven solution to render materials with complex appearance. A typical capture contains tens of thousands of images of a material sample under varying viewing and lighting conditions. While capable of faithfully recording complex light interactions in the material, the main drawback is the massive memory requirement, both for storing and rendering, making effective compression of BTF data a critical component in practical applications. Common compression schemes used in practice are based on matrix factorization techniques, which preserve the discrete format of the original dataset. While this approach generalizes well to different materials, rendering with the compressed dataset still relies on interpolating between the closest samples. Depending on the material and the angular resolution of the BTF, this can lead to blurring and ghosting artefacts. An alternative approach uses analytic model fitting to approximate the BTF data, using continuous functions that naturally interpolate well, but whose expressive range is often not wide enough to faithfully recreate materials with complex non‐local lighting effects (subsurface scattering, inter‐reflections, shadowing and masking…). In light of these observations, we propose a neural network‐based BTF representation inspired by autoencoders: our encoder compresses each texel to a small set of latent coefficients, while our decoder additionally takes in a light and view direction and outputs a single RGB vector at a time. This allows us to continuously query reflectance values in the light and view hemispheres, eliminating the need for linear interpolation between discrete samples. We train our architecture on fabric BTFs with a challenging appearance and compare to standard PCA as a baseline. We achieve competitive compression ratios and high‐quality interpolation/extrapolation without blurring or ghosting artifacts.

show abstract

“…Assigning materials to a complex scene is a laborious process [Chen et al 2015;Zsolnai-Fehér et al 2018]. We can leverage the fact that the distances in our learned feature space correlate with human perception of similarity to provide controllable material suggestions.…”

Section: Materials Suggestionsmentioning

confidence: 99%

“…Given the large number of parameters involved in our perception of materials, many works have focused on individual attributes (such as the perception of gloss [Pellacini et al 2000;Wills et al 2009], or translucency [Gkioulekas et al 2015]), while others have focused on particular applications like material synthesis [Zsolnai-Fehér et al 2018], editing [Serrano et al 2016], or filtering [Jarabo et al 2014]. However, the fundamentally difficult problem of establishing a similarity measure for material appearance remains an open problem.…”

Section: Introductionmentioning

confidence: 99%

A similarity measure for material appearance

et al. 2019

View full text Add to dashboard Cite

We present a model to measure the similarity in appearance between different materials, which correlates with human similarity judgments. We first create a database of 9,000 rendered images depicting objects with varying materials, shape and illumination. We then gather data on perceived similarity from crowdsourced experiments; our analysis of over 114,840 answers suggests that indeed a shared perception of appearance similarity exists. We feed this data to a deep learning architecture with a novel loss function, which learns a feature space for materials that correlates with such perceived appearance similarity. Our evaluation shows that our model outperforms existing metrics. Last, we demonstrate several applications enabled by our metric, including appearance-based search for material suggestions, database visualization, clustering and summarization, and gamut mapping.

show abstract

Gaussian material synthesis

Cited by 29 publications

References 48 publications

Offline Deep Importance Sampling for Monte Carlo Path Tracing

Offline Deep Importance Sampling for Monte Carlo Path Tracing

Neural BTF Compression and Interpolation

A similarity measure for material appearance

Contact Info

Product

Resources

About