Automatic Differentiable Procedural Modeling

Gaillard, Mathieu; Krs, Vojtĕch; Gori, Giorgio; Mĕch, Radomír; Beneš, Bedřich

doi:10.1111/cgf.14475

Cited by 7 publications

(4 citation statements)

References 87 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While there is some work on local editing from program analysis [JPCS18; LSL*19], none of these methods allow users to control geometric edits directly while maintaining a working program. In concurrent work to ours, Gaillard et al [GKG*22] enable direct editing at the granularity of objects, and directly tackle exploring ambiguous edits made possible in part by a fast objective function. In contrast, our work enables editing at a vertex granularity while relying on heuristics to address ambiguity.…”

Section: Background and Related Workmentioning

confidence: 99%

Differentiable 3D CAD Programs for Bidirectional Editing

Cascaval

Shalah

Quinn

et al. 2022

Computer Graphics Forum

View full text Add to dashboard Cite

Modern CAD tools represent 3D designs not only as geometry, but also as a program composed of geometric operations, each of which depends on a set of parameters. Program representations enable meaningful and controlled shape variations via parameter changes. However, achieving desired modifications solely through parameter editing is challenging when CAD models have not been explicitly authored to expose select degrees of freedom in advance. We introduce a novel bidirectional editing system for 3D CAD programs. In addition to editing the CAD program, users can directly manipulate 3D geometry and our system infers parameter updates to keep both representations in sync. We formulate inverse edits as a set of constrained optimization objectives, returning plausible updates to program parameters that both match user intent and maintain program validity. Our approach implements an automatically differentiable domain‐specific language for CAD programs, providing derivatives for this optimization to be performed quickly on any expressed program. Our system enables rapid, interactive exploration of a constrained 3D design space by allowing users to manipulate the program and geometry interchangeably during design iteration. While our approach is not designed to optimize across changes in geometric topology, we show it is expressive and performant enough for users to produce a diverse set of design variants, even when the CAD program contains a relatively large number of parameters.

show abstract

Section: Background and Related Workmentioning

confidence: 99%

Differentiable 3D CAD Programs for Bidirectional Editing

Cascaval

Shalah

Quinn

et al. 2022

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…For code that generates a visual output, manipulation is extremely important to enable customization and iterative design. While recent work describes techniques to optimize code based on the direct manipulation of the visual output [HLC19, CSQ*22, GKG*22], these methods are limited in the types of variations they enable, only allowing the program parameters to change, but not its structure. Furthermore, existing techniques fundamentally struggle to infer the user intent, since direct visual manipulation is a partial (and therefore ambiguous) specification.…”

Section: Discussionmentioning

confidence: 99%

Neurosymbolic Models for Computer Graphics

Ritchie

Guerrero

Jones

et al. 2023

Computer Graphics Forum

View full text Add to dashboard Cite

Procedural models (i.e. symbolic programs that output visual data) are a historically‐popular method for representing graphics content: vegetation, buildings, textures, etc. They offer many advantages: interpretable design parameters, stochastic variations, high‐quality outputs, compact representation, and more. But they also have some limitations, such as the difficulty of authoring a procedural model from scratch. More recently, AI‐based methods, and especially neural networks, have become popular for creating graphic content. These techniques allow users to directly specify desired properties of the artifact they want to create (via examples, constraints, or objectives), while a search, optimization, or learning algorithm takes care of the details. However, this ease of use comes at a cost, as it's often hard to interpret or manipulate these representations. In this state‐of‐the‐art report, we summarize research on neurosymbolic models in computer graphics: methods that combine the strengths of both AI and symbolic programs to represent, generate, and manipulate visual data. We survey recent work applying these techniques to represent 2D shapes, 3D shapes, and materials & textures. Along the way, we situate each prior work in a unified design space for neurosymbolic models, which helps reveal underexplored areas and opportunities for future research.

show abstract

“…A more general approach has also been studied. In the work [52], the authors focus on an interactive procedural generation tool that allows for greater user control over the generated output. The user then modifies the output interactively, and the modifications are passed back to the procedural model as its parameters by solving the inverse procedural modeling problem.…”

Section: Inverse Procedural Modelingmentioning

confidence: 99%

Single-View 3D Reconstruction via Differentiable Rendering and Inverse Procedural Modeling

Garifullin,

Maiorov,

Frolov

et al. 2024

Symmetry

View full text Add to dashboard Cite

Three-dimensional models, reconstructed from real-life objects, are extensively used in virtual and mixed reality technologies. In this paper we propose an approach to 3D model reconstruction via inverse procedural modeling and describe two variants of this approach. The first option is to fit a set of input parameters using a genetic algorithm. The second option allows us to significantly improve precision by using gradients within the memetic algorithm, differentiable rendering, and differentiable procedural generators. We demonstrate the results of our work on different models, including trees, which are complex objects that most existing methods cannot reconstruct. In our work, we see two main contributions. First, we propose a method to join differentiable rendering and inverse procedural modeling. This gives us the ability to reconstruct 3D models more accurately than existing approaches when few input images are available, even for a single image. Second, we combine both differentiable and non-differentiable procedural generators into a single framework that allows us to apply inverse procedural modeling to fairly complex generators. We show that both variants of our approach can be useful: the differentiable one is more precise but puts limitations on the procedural generator, while the one based on genetic algorithms can be used with any existing generator. The proposed approach uses information about the symmetry and structure of the object to achieve high-quality reconstruction from a single image.

show abstract

Automatic Differentiable Procedural Modeling

Cited by 7 publications

References 87 publications

Differentiable 3D CAD Programs for Bidirectional Editing

Differentiable 3D CAD Programs for Bidirectional Editing

Neurosymbolic Models for Computer Graphics

Single-View 3D Reconstruction via Differentiable Rendering and Inverse Procedural Modeling

Contact Info

Product

Resources

About