Highlight-aware two-stream network for single-image SVBRDF acquisition

Guo, Jie; Lai, Shuichang; Tao, Chengzhi; Cai, Yuelong; Wang, Lei; Guo, Yanwen; Yan, Ling‐Qi

doi:10.1145/3476576.3476693

Cited by 8 publications

(8 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[GSH*20], Guo et al . [GLT*21] and Zhao et al . [ZWX*20] on both synthetic images and captured photos (Section 5.1).…”

Section: Resultsmentioning

confidence: 99%

“…[GLD*19] and Guo et al . [GLT*21] produce ‘polluted’ SVBRDF maps that are highly affected by highlight regions, while Guo et al . [GSH*20] produce blurred results that lack detailed structure in SVBRDF maps.…”

Section: Resultsmentioning

confidence: 99%

“…Guo et al . [GLT*21] proposed a new convolution variant called highlight‐aware convolution to ‘guess’ the saturated pixels (specular highlight area) by the unsaturated area surrounded, making the extracted features more uniform. Their work can well handle images with intense highlights.…”

Section: Related Workmentioning

confidence: 99%

“…Recent works have shown that it is possible to recover the SVBRDFs from a few photos, or even a single image, of the material [DAD*18, LSC18, GLD*19, GSH*20, GLT*21]. These lightweight methods used deep neural networks to capture four SVBRDF maps (diffuse and specular albedo, normal map and roughness parameters) from photographs of a material.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

SVBRDF Recovery from a Single Image with Highlights Using a Pre‐trained Generative Adversarial Network

Wen

Wang

Zhang

et al. 2022

Computer Graphics Forum

Self Cite

View full text Add to dashboard Cite

Spatially varying bi-directional reflectance distribution functions (SVBRDFs) are crucial for designers to incorporate new materials in virtual scenes, making them look more realistic. Reconstruction of SVBRDFs is a long-standing problem. Existing methods either rely on an extensive acquisition system or require huge datasets, which are non-trivial to acquire. We aim to recover SVBRDFs from a single image, without any datasets. A single image contains incomplete information about the SVBRDF, making the reconstruction task highly ill-posed. It is also difficult to separate between the changes in colour that are caused by the material and those caused by the illumination, without the prior knowledge learned from the dataset. In this paper, we use an unsupervised generative adversarial neural network (GAN) to recover SVBRDFs maps with a single image as input. To better separate the effects due to illumination from the effects due to the material, we add the hypothesis that the material is stationary and introduce a new loss function based on Fourier coefficients to enforce this stationarity. For efficiency, we train the network in two stages: reusing a trained model to initialize the SVBRDFs and fine-tune it based on the input image. Our method generates high-quality SVBRDFs maps from a single input photograph, and provides more vivid rendering results compared to the previous work. The two-stage training boosts runtime performance, making it eight times faster than the previous work.

show abstract

“…[GSH*20], Guo et al . [GLT*21] and Zhao et al . [ZWX*20] on both synthetic images and captured photos (Section 5.1).…”

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

SVBRDF Recovery from a Single Image with Highlights Using a Pre‐trained Generative Adversarial Network

Wen

Wang

Zhang

et al. 2022

Computer Graphics Forum

Self Cite

View full text Add to dashboard Cite

show abstract

“…Aittala et al [AAL16] proposed using a single flash image of a stationary material to reconstruct a patch of it through neural guided optimization. Recently, deep learning was used to improve single [LDPT17,DAD∗18,HDMR21, GLT∗21, ZK21] and few‐images [DAD∗19, GSH∗20, GLD∗19, YDPG21] material acquisition. These methods recover 2D material maps based on an analytical BRDF model e.g.…”

Section: Related Workmentioning

confidence: 99%

Controlling Material Appearance by Examples

Hašan

Guerrero

et al. 2022

Computer Graphics Forum

View full text Add to dashboard Cite

Figure 1: We propose a method to control and improve the appearance of an existing material (left) by transferring the appearance of materials in one or multiple target photo(s) (center) to the existing material. The augmented material (right) combines the coarse structure from the original material with the fine-scale appearance of the target(s) and preserve the input tileability. Our method can also transfer appearance from materials from different types by spatial control. This enables a simple workflow to make existing materials more realistic using readily-available images or photos.

show abstract

An attention-embedded GAN for SVBRDF recovery from a single image

2023

View full text Add to dashboard Cite

Learning-based approaches have made substantial progress in capturing spatially-varying bidirectional reflectance distribution functions (SVBRDFs) from a single image with unknown lighting and geometry. However, most existing networks only consider per-pixel losses which limit their capability to recover local features such as smooth glossy regions. A few generative adversarial networks use multiple discriminators for different parameter maps, increasing network complexity. We present a novel end-to-end generative adversarial network (GAN) to recover appearance from a single picture of a nearly-flat surface lit by flash. We use a single unified adversarial framework for each parameter map. An attention module guides the network to focus on details of the maps. Furthermore, the SVBRDF map loss is combined to prevent paying excess attention to specular highlights. We demonstrate and evaluate our method on both public datasets and real data. Quantitative analysis and visual comparisons indicate that our method achieves better results than the state-of-the-art in most cases.

show abstract

Highlight-aware two-stream network for single-image SVBRDF acquisition

Cited by 8 publications

References 37 publications

SVBRDF Recovery from a Single Image with Highlights Using a Pre‐trained Generative Adversarial Network

SVBRDF Recovery from a Single Image with Highlights Using a Pre‐trained Generative Adversarial Network

Controlling Material Appearance by Examples

An attention-embedded GAN for SVBRDF recovery from a single image

Contact Info

Product

Resources

About