pix2xray: converting RGB images into X-rays using generative adversarial networks

Haiderbhai, Mustafa; Ledesma, Sergio; Lee, Sing Chun; Seibold, Matthias; Fürnstahl, Philipp; Navab, Nassir; Fallavollita, Pascal

doi:10.1007/s11548-020-02159-2

Cited by 8 publications

(2 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In another study [36], to generate new modalities, Haiderbhai et al introduced a novel architecture based on the pix2pix model. They proposed a method of synthetic X-ray generation using conditional generative adversarial networks and created triplets for X-ray, pose, and RGB images of natural hand poses sampled from the NYU hand pose dataset .…”

Section: Image-to-image Translationmentioning

confidence: 99%

A Survey on GAN-Based Data Augmentation for Hand Pose Estimation Problem

et al. 2022

View full text Add to dashboard Cite

Deep learning solutions for hand pose estimation are now very reliant on comprehensive datasets covering diverse camera perspectives, lighting conditions, shapes, and pose variations. While acquiring such datasets is a challenging task, several studies circumvent this problem by exploiting synthetic data, but this does not guarantee that they will work well in real situations mainly due to the gap between the distribution of synthetic and real data. One recent popular solution to the domain shift problem is learning the mapping function between different domains through generative adversarial networks. In this study, we present a comprehensive study on effective hand pose estimation approaches, which are comprised of the leveraged generative adversarial network (GAN), providing a comprehensive training dataset with different modalities. Benefiting from GAN, these algorithms can augment data to a variety of hand shapes and poses where data manipulation is intuitively controlled and greatly realistic. Next, we present related hand pose datasets and performance comparison of some of these methods for the hand pose estimation problem. The quantitative and qualitative results indicate that the state-of-the-art hand pose estimators can be greatly improved with the aid of the training data generated by these GAN-based data augmentation methods. These methods are able to beat the baseline approaches with better visual quality and higher values in most of the metrics (PCK and ME) on both the STB and NYU datasets. Finally, in conclusion, the limitation of the current methods and future directions are discussed.

show abstract

Section: Image-to-image Translationmentioning

confidence: 99%

A Survey on GAN-Based Data Augmentation for Hand Pose Estimation Problem

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Alternatively, fast analytical simulations can also be used to generate training data in a controlled manner. For example, Haiderbhai et al [8] proposed a method based on a generative adversarial network (GAN), a machine learning approach that can be used to create synthetic images. Images simulated using gVirtualXRay, the same X-ray generator as in our registration framework, are used to create a large database to train the GAN.…”

Section: Context: Artefacts In Ctmentioning

confidence: 99%