Consistent Scene Editing by Progressive Difference Images

Abstract

Even though much research was dedicated to the acceleration of consistent, progressive light transport simulations, the computation of fully converged images is still very time‐consuming. This is problematic, as for the practical use in production pipelines, the rapid editing of lighting effects is important. While previous approaches restart the simulation with every scene manipulation, we make use of the coherence between frames before and after a modification in order to accelerate convergence of the context that remained similar. This is especially beneficial if a scene is edited that has already been converging for a long time, because much of the previous result can be reused, e.g., sharp caustics cast or received by the unedited scene parts. In its essence, our method performs the scene modification stochastically by predicting and accounting for the difference image. In addition, we employ two heuristics to handle cases in which stochastic removal is likely to lead to strong noise. Typical scene interactions can be broken down into object adding and removal, material substitution, camera movement and light editing, which we all examine in a number of test scenes both qualitatively and quantitatively. As we focus on caustics, we chose stochastic progressive photon mapping as the underlying light transport algorithm. Further, we show preliminary results of bidirectional path tracing and vertex connection and merging.

Residual path integrals for re‐rendering

“…Scene editing and re‐rendering. Our problem setting is related to previous work on efficient scene re‐rendering [GG15]. In particular, control variates and correlated sampling have been applied to re‐rendering previously [RJN16].…”

Dual attention autoencoder for all-weather outdoor lighting estimation

“…Outdoor illumination estimation is beneficial to many vision tasks ranging from scene understanding [1], reconstruction [2] to image editing [3]. It also has many potential applications including movie production and augmented reality (AR).…”