Voxel-Based 3D Object Reconstruction from Single 2D Image Using Variational Autoencoders

Tahir, Rohan; Sargano, Allah Bux; Habib, Zulfiqar

doi:10.3390/math9182288

Cited by 21 publications

(10 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has also been shown that VAE models are capable of learning representations with disentangled factors (Higgins et al, 2016a) due to the isotropic Gaussian priors on the latent variable, the known power of the Bayesian models. The better performance of VAE compared to AE models has also been shown previously in other applications such as anomaly detection, object identification, and BCIs (Dai et al, 2019;Tahir et al, 2021;Zhou et al, 2021). As an additional point, comparing the results of VAE with SVAE, suggests the added value of supervised learning in training better models.…”

Section: Impact Of Variational Inference In Feature Learningsupporting

confidence: 72%

Subject-invariant feature learning for mTBI identification using LSTM-based variational autoencoder with adversarial regularization

Salsabilian

Najafizadeh

2022

Front. Signal Process.

View full text Add to dashboard Cite

Developing models for identifying mild traumatic brain injury (mTBI) has often been challenging due to large variations in data from subjects, resulting in difficulties for the mTBI-identification models to generalize to data from unseen subjects. To tackle this problem, we present a long short-term memory-based adversarial variational autoencoder (LSTM-AVAE) framework for subject-invariant mTBI feature extraction. In the proposed model, first, an LSTM variational autoencoder (LSTM-VAE) combines the representation learning ability of the variational autoencoder (VAE) with the temporal modeling characteristics of the LSTM to learn the latent space representations from neural activity. Then, to detach the subject’s individuality from neural feature representations, and make the model proper for cross-subject transfer learning, an adversary network is attached to the encoder in a discriminative setting. The model is trained using the 1 held-out approach. The trained encoder is then used to extract the representations from the held-out subject’s data. The extracted representations are then classified into normal and mTBI groups using different classifiers. The proposed model is evaluated on cortical recordings of Thy1-GCaMP6s transgenic mice obtained via widefield calcium imaging, prior to and after inducing injury. In cross-subject transfer learning experiment, the proposed LSTM-AVAE framework achieves classification accuracy results of 95.8% and 97.79%, without and with utilizing conditional VAE (cVAE), respectively, demonstrating that the proposed model is capable of learning invariant representations from mTBI data.

show abstract

Section: Impact Of Variational Inference In Feature Learningsupporting

confidence: 72%

Subject-invariant feature learning for mTBI identification using LSTM-based variational autoencoder with adversarial regularization

Salsabilian

Najafizadeh

2022

Front. Signal Process.

View full text Add to dashboard Cite

show abstract

“…However, it leads to the inability of the body model to model the human body in clothing. Therefore, nonparametric methods such as hulls [38], point clouds [39], triangular meshes [40], and voxel grids [41] were used for 3D human body reconstruction. They can predict shape representations directly from images.…”

Section: Human Body Reconstruction From Imagesmentioning

confidence: 99%

Fashion Intelligence in the Metaverse: Promise and Future Prospects

Zhang

Shi

et al. 2023

Preprint

View full text Add to dashboard Cite

With the development of artificial intelligence (AI) and the constraints on offline activities imposed due to the sudden outbreak of the COVID epidemic, the Metaverse has recently attracted significant research attention from both academia and industrial practitioners. Fashion, as an expression of a consumer’s aesthetics and personality, has enormous economic potential in both the real world and the Metaverse. In this research, we provide a comprehensive survey of two of the most important components of fashion in the Metaverse: virtual digital humans, and tasks related to fashion items. We survey state-of-the-art articles from 2007 to the present and provide a new taxonomy of extant research topics based on these articles. We also highlight the applications of these topics in the Metaverse from the perspectives of designers and consumers. Finally, we describe possible scenes involving fashion in the Metaverse. The current challenges and open issues related to the fashion industry in the Metaverse are also discussed in order to provide guidance for fashion practitioners, and to shed some light on the future development of fashion AI in the Metaverse.

show abstract

“…Several approaches focus on generating voxelized output representations. V3DOR [ 14 ] applied autoencoders and variational autoencoders to generate a smoother and high-resolution 3D model. The encoder aims to learn latent representation from the image and the decoder tries to obtain corresponding 3D voxels.…”

Section: Related Workmentioning

confidence: 99%

A Single Stage and Single View 3D Point Cloud Reconstruction Network Based on DetNet

Zhu

2022

Sensors

View full text Add to dashboard Cite

It is a challenging problem to infer objects with reasonable shapes and appearance from a single picture. Existing research often pays more attention to the structure of the point cloud generation network, while ignoring the feature extraction of 2D images and reducing the loss in the process of feature propagation in the network. In this paper, a single-stage and single-view 3D point cloud reconstruction network, 3D-SSRecNet, is proposed. The proposed 3D-SSRecNet is a simple single-stage network composed of a 2D image feature extraction network and a point cloud prediction network. The single-stage network structure can reduce the loss of the extracted 2D image features. The 2D image feature extraction network takes DetNet as the backbone. DetNet can extract more details from 2D images. In order to generate point clouds with better shape and appearance, in the point cloud prediction network, the exponential linear unit (ELU) is used as the activation function, and the joint function of chamfer distance (CD) and Earth mover’s distance (EMD) is used as the loss function of 3DSSRecNet. In order to verify the effectiveness of 3D-SSRecNet, we conducted a series of experiments on ShapeNet and Pix3D datasets. The experimental results measured by CD and EMD have shown that 3D-SSRecNet outperforms the state-of-the-art reconstruction methods.

show abstract

Voxel-Based 3D Object Reconstruction from Single 2D Image Using Variational Autoencoders

Cited by 21 publications

References 38 publications

Subject-invariant feature learning for mTBI identification using LSTM-based variational autoencoder with adversarial regularization

Subject-invariant feature learning for mTBI identification using LSTM-based variational autoencoder with adversarial regularization

Fashion Intelligence in the Metaverse: Promise and Future Prospects

A Single Stage and Single View 3D Point Cloud Reconstruction Network Based on DetNet

Contact Info

Product

Resources

About