3D-Aware Semantic-Guided Generative Model for Human Synthesis

Zhang, Jichao; Sangineto, Enver; Tang, Hao; Siarohin, Aliaksandr; Zhong, Zhun; Sebe, Nicu; Wang, Wei

doi:10.1007/978-3-031-19784-0_20

Cited by 17 publications

(3 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, most of these surveys have adopted a methodological perspective, which is useful for researchers who want to understand the underlying principles and algorithms of image synthesis, but not for practitioners who want to apply image synthesis techniques to solve specific problems in various domains [18], [19]. This paper provides a task-oriented review of low-level controllable image synthesis, excluding human subjects [20], [21], [22], [23].…”

Section: Introductionmentioning

confidence: 99%

Enhancing image resolution of confocal fluorescence microscopy with deep learning

Huang

Yao

et al. 2023

PhotoniX

View full text Add to dashboard Cite

Super-resolution optical imaging is crucial to the study of cellular processes. Current super-resolution fluorescence microscopy is restricted by the need of special fluorophores or sophisticated optical systems, or long acquisition and computational times. In this work, we present a deep-learning-based super-resolution technique of confocal microscopy. We devise a two-channel attention network (TCAN), which takes advantage of both spatial representations and frequency contents to learn a more precise mapping from low-resolution images to high-resolution ones. This scheme is robust against changes in the pixel size and the imaging setup, enabling the optimal model to generalize to different fluorescence microscopy modalities unseen in the training set. Our algorithm is validated on diverse biological structures and dual-color confocal images of actin-microtubules, improving the resolution from ~ 230 nm to ~ 110 nm. Last but not least, we demonstrate live-cell super-resolution imaging by revealing the detailed structures and dynamic instability of microtubules.

show abstract

Section: Introductionmentioning

confidence: 99%

Enhancing image resolution of confocal fluorescence microscopy with deep learning

Huang

Yao

et al. 2023

PhotoniX

View full text Add to dashboard Cite

show abstract

“…These techniques can largely be categorized as (a) encoder-based methods [4,8,32,38,42,42,51,52,52,54] in which a neural network encoder is trained to project an input image to the latent space of the generator; (b) optimization-based methods [1,2,13,14,22,25,41,50] where the latent code is recovered via optimizing loss functions between the generator output and a target image; and (c) hybrid methods [5,7,43,65] which combine both approaches. Some recent works [31,48,62] have also investigated 3D-aware GAN inversion. Unlike previous methods focusing on single-image inversion, we consider multi-shot (or video) inversion.…”

Section: Related Workmentioning

confidence: 99%

In-N-Out: Face Video Inversion and Editing with Volumetric Decomposition

Xu¹,

Shu²,

Smith³

et al. 2023

Preprint

View full text Add to dashboard Cite

“…On the other hand, although [25,26,141] can synthesize high-resolution images with controllable viewpoints, their learned representations are highly entangled, making it challenging for users to manipulate specific attributes of the generated images. To alleviate this issue, some recent studies have explored the use of prior knowledge to explicitly control the semantic attributes of the generated faces, including 3D facial parameters [142,143], semantic maps [144,145], one-hot attribute vectors [146] and the learned disentangled latent codes [147]. However, the area of text-guided 3D-aware editing in generative NeRF remains relatively unexplored.…”

Section: D-aware Generative Modelsmentioning

confidence: 99%

Deep generative modeling for image synthesis and manipulation

View full text Add to dashboard Cite

pletion of this thesis. Their invaluable support, guidance, and encouragement have been indispensable during the entirety of my research journey. This thesis is a testament to their collective efforts and support. First and foremost, I wish to express my deepest gratitude to my supervisor Prof. Shijian Lu, for his continuous guidance, patience, and invaluable advice. His insightful feedback and dedication to my academic and personal growth have played a crucial role in shaping this research project. Without his patient guidance and thoughtful counsel, I would not have been able to accomplish my research and successfully conclude my studies. I also want to express my profound gratitude to my collaborators, especially Dr. Fangneng Zhan, for their insightful input, helpful advice, and unwavering support during my research endeavors. Furthermore, I extend my gratitude to all members of Prof. Lu's team, my friends in the Alibaba Talent Programme and Media and Interactive Computing Lab (MICL), for their generous assistance and encouragement, which have made my academic journey and time spent in Singapore truly memorable. I would like to express my gratitude to my fellow members of the Vision Intelligent Team at DAMO Academy, Alibaba, specifically my team leaders, Ms. Feiyang Ma and Ms. Miaomiao Cui, for supporting me with computational resources, technical assistance, and insightful suggestions. My heartfelt thanks go to the School of Computer Science and Engineering, Nanyang Technological University (NTU), for providing me with the necessary resources and a conducive environment for my research. I also would like to acknowledge Alibaba-NTU Singapore Joint Research Institute for offering me this Industrial Post-Graduate Program and for their support during my studies. xi xii Special thanks to my former colleague, Dr. Pan Liu, for his continuous support and invaluable assistance throughout the process of my Ph.D. application. Last but not the least, I am forever grateful to my parents, for their unconditional love, constant encouragement, and unwavering belief in my capabilities throughout my life. Their support and understanding have been the bedrock of my academic pursuits.

show abstract

3D-Aware Semantic-Guided Generative Model for Human Synthesis

Cited by 17 publications

References 58 publications

Enhancing image resolution of confocal fluorescence microscopy with deep learning

Enhancing image resolution of confocal fluorescence microscopy with deep learning

In-N-Out: Face Video Inversion and Editing with Volumetric Decomposition

Deep generative modeling for image synthesis and manipulation

Contact Info

Product

Resources

About