Learning to reconstruct botanical trees from single images

Li, Bosheng; Kałużny, Jacek; Klein, Jonathan; Michels, Dominik L.; Pałubicki, Wojciech; Beneš, Bedřich; Pirk, Sören

doi:10.1145/3478513.3480525

Cited by 35 publications

(12 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…According to this image, a 3D model of the tree is assembled from a given set of branches. The work [Li21] also uses only one image for creating a model, but fully automates this process by using three neural networks: the first is used to segment the image, the second to form an approximate representation of the tree in the form of radial bounding volumes, and the third to determine the type of plant. A plant species is defined as a set of parameters for a specific procedural generator, also described in the paper.…”

Section: Image-based Modelingmentioning

confidence: 99%

“…Semantic masks are used for comparison, where each pixel belongs to one of three categories: branch, foliage, background. To obtain such a mask from the source image, a neural network can be used similarly to [Li21], and in simple cases, we can get it based only on pixel color (green corresponds to leaves, brown or gray -to branches and trunk).…”

Section: Similarity Functionmentioning

confidence: 99%

See 1 more Smart Citation

Fitting Parameters for Procedural Plant Generation

Garifullin¹,

Shcherbakov²,

Фролов³

2022

CSRN

View full text Add to dashboard Cite

We propose a novel method to obtain a 3D model of a tree based on a single input image by fitting parameters for some procedural plant generator. Unlike other methods, our approach can work with any plant generator, treating it as a black-box function. It is also possible to specify the desired characteristics of the plant, such as the geometric complexity of the model or its size. We propose a similarity function between the given image and generated model, that better catches the significant differences between tree shapes. To find the appropriate parameter set, we use a specific variant of a genetic algorithm designed for this purpose to maximize similarity function. This approach can greatly simplify the artist's work. We demonstrate the results of our algorithm with several procedural generators, from a very simple to a fairly advanced one.

show abstract

Section: Image-based Modelingmentioning

confidence: 99%

Section: Similarity Functionmentioning

confidence: 99%

Fitting Parameters for Procedural Plant Generation

Garifullin¹,

Shcherbakov²,

Фролов³

2022

CSRN

View full text Add to dashboard Cite

show abstract

“…The latter is demonstrated by the predictive accuracy of the parameters of our TSN, even in difficult conditions, as reported in Section 5.1. Although procedural modelling has only recently been explored in the field of deep learning and artificial intelligence [80,81,82], there are, as yet, few specific approaches that examine prediction parameters for 3D non-linear content generation, either plants, in general, or trees [40,50], in particular. Moreover, there are no approaches that use very rudimentary input data such as sketches.…”

Section: Discussion and Limitationsmentioning

confidence: 99%

“…There is an interesting inverse procedural modeling approach, introduced by [49], that use deep learning to detect an L-system from an input tree image. Another approach, provided by [50], use a combination of deep learning and procedural modeling algorithms to extract a 3D tree model from a single image. This method is based on three DNNs that, starting form an input photograph of a tree, mask out the tree, identify the tree type, and extract the tree Radial Bounding Volume (RBV), respectively.…”

Section: Procedural Modelingmentioning

confidence: 99%

TreeSketchNet: From Sketch To 3D Tree Parameters Generation

Manfredi¹,

Capece²,

Erra³

et al. 2022

Preprint

View full text Add to dashboard Cite

3D modeling of non-linear objects from stylized sketches is a challenge even for experts in computer graphics. The extrapolation of objects parameters from a stylized sketch is a very complex and cumbersome task. In the present study, we propose a broker system that mediates between the modeler and the 3D modelling software and can transform a stylized sketch of a tree into a complete 3D model. The input sketches do not need to be accurate or detailed, and only need to represent a rudimentary outline of the tree that the modeler wishes to 3D-model. Our approach is based on a well-defined Deep Neural Network (DNN) architecture, we called TreeSketchNet (TSN), based on convolutions and able to generate Weber and Penn [1] parameters that can be interpreted by the modelling software to generate a 3D model of a tree starting from a simple sketch. The training dataset consists of synthetically-generated sketches that are associated with Weber-Penn parameters generated by a dedicated Blender modelling software add-on. The accuracy of the proposed method is demonstrated by testing the TSN with both synthetic and hand-made sketches. Finally, we provide a qualitative analysis of our results, by evaluating the coherence of the predicted parameters with several distinguishing features.

show abstract

“…Moreover, sketch-based modeling techniques allow artists to produce plant models interactively and in more nuanced ways [Ijiri et al 2006;Okabe et al 2007;Wither et al 2009]. Alternative approaches attempt to reconstruct plant models automatically either from images [Li et al 2021;Tan et al 2008Tan et al , 2007, videos [Li et al 2011a], or scanned 3D point clouds [Livny et al 2011;Xie et al 2016]. Only just recently, several approaches also focus on the dynamic and realistic behavior of plant models, including growth [Longay et al 2012;Pirk et al 2012a], the interaction with wind or fire , or as established through realistic materials [Wang et al 2017;Zhao and Barbič 2013].…”

Section: Related Workmentioning

confidence: 99%

Procedural Urban Forestry

et al. 2022

Self Cite

View full text Add to dashboard Cite

The placement of vegetation plays a central role in the realism of virtual scenes. We introduce procedural placement models (PPMs) for vegetation in urban layouts. PPMs are environmentally sensitive to city geometry and allow identifying plausible plant positions based on structural and functional zones in an urban layout. PPMs can either be directly used by defining their parameters or learned from satellite images and land register data. This allows us to populate urban landscapes with complex 3D vegetation and enhance existing approaches for generating urban landscapes. Our framework’s effectiveness is shown through examples of large-scale city scenes and close-ups of individually grown tree models. We validate the results generated with our framework with a perceptual user study and its usability based on urban scene design sessions with expert users.

show abstract

Learning to reconstruct botanical trees from single images

Cited by 35 publications

References 70 publications

Fitting Parameters for Procedural Plant Generation

Fitting Parameters for Procedural Plant Generation

TreeSketchNet: From Sketch To 3D Tree Parameters Generation

Procedural Urban Forestry

Contact Info

Product

Resources

About