Self-calibrating Photometric Stereo by Neural Inverse Rendering

MEGANE: Morphable Eyeglass and Avatar Network

“…( 7), which encodes face information. We compute specular feature A spec at every point on primitives based on normal, light and view directions with a specular BRDF parameterized as Spherical Gaussians [31] with three different lobes. We observe that explicitly conditioning specular reflection significantly improves fidelity of relighting and generalization to various frame materials.…”

DANI-Net: Uncalibrated Photometric Stereo by Differentiable Shadow Handling, Anisotropic Reflectance Modeling, and Neural Inverse Rendering

“…However, these methods assume the light intensity distributing in a pre-defined range (i.e., [0.2, 2]) and solve UPS in two-stage, making them suffer from the data bias (between synthetic training data and real-world ones) and the accumulating errors. Recently, SCPS-NIR [26] utilizes the neural inverse rendering method to jointly optimize light and surface normal in an unsupervised manner based on local reflectance information, free from data bias and accumulating errors. The proposed DANI-Net also addresses UPS in an unsupervised manner, but it differs from all previous works in two aspects.…”

“…2) Our method introduces the anisotropic reflectance model in solving UPS, which improves the performance on general materials. Besides, as compared with [26] that calculates shadow maps by image binarization and fixes them during training, our method computes shadow maps from shapes and constantly updates them during training.…”

“…However, GBR ambiguity is alleviated on a non-Lambertian surface [13]. Therefore, recent advances in UPS (e.g., [26,56]) adopt the isotropic reflectance model accounting for non-Lambertian effects to solve UPS. Nonetheless, such a model restricts methods' performance on objects with more general (e.g., anisotropic) materials, while modeling general reflectance is challenging due to extra unknowns, which eventually make UPS intractable.…”

“…DANI-Net adopts an anisotropic reflectance model. The state-of-the-art UPS method SCPS-NIR [26] is compared in this figure. As can be observed, the proposed DANI-Net produces a smoother and more realistic shadow map of copper BUNNY thanks to the differentiable shadow handling and renders a more realistic copper BALL image and Bidirectional Reflectance Distribution Function (BRDF) sphere (of the reference point) due to the anisotropic reflectance modeling.…”