Semantics-Guided Exploration of Latent Spaces for Shape Synthesis

Abstract

We introduce an approach to incorporate user guidance into shape synthesis approaches based on deep networks. Synthesis networks such as auto-encoders are trained to encode shapes into latent vectors, effectively learning a latent shape space that can be sampled for generating new shapes. Our main idea is to allow users to start an exploratory process of the shape space with the use of high-level semantic keywords. Specifically, the user inputs a set of keywords that describe the general attributes of the shape to be generated, e.g., "four legs" for a chair. Next, we map the keywords to a subspace of the latent space that captures the shapes possessing the specified attributes. The user then explores only the subspace to search for shapes that satisfy the design goal, in a process similar to using a parametric shape model. Our exploratory approach allows users to model shapes at a high-level without the need of advanced artistic skills, in contrast to existing methods that allow to guide the synthesis with sketching or partial modeling of a shape. Our technical contribution to enable this exploration-based approach is the introduction of a label regression neural network coupled with a shape synthesis neural network. The label regression network takes the user-provided keywords and maps them to the corresponding subspace of the latent space, where the subspace is modeled as a set of distributions for the dimensions of the latent space. We show that our method allows users to efficiently explore the shape space and generate a variety of shapes with selected high-level attributes. iii I would also like to thank my defense committee Dr. David Mould, Dr. Yongyi Mao and Dr. Matthew Holden for their valuable and useful critiques towards the thesis. Special thanks to my fellow lab member Yanran Guan, who helped me in labeling the training dataset for this research. My sincere thanks goes to all members of GIGL lab for their consistent motivation and support in the past two years. I am also grateful to the School of Computer Science, Carleton University for accepting me as a research student and facilitating me with financial assistance to complete my degree. At last, I would like to thank my parents, sister, family members and my beloved husband for their unconditional love and support that encouraged me to complete my research.