Dynamic GAN for high-quality sign language video generation from skeletal poses using generative adversarial networks

Abstract

The emergence of unsupervised generative models has resulted in greater performance in image and video generation tasks. However, existing generative models pose huge challenges in high-quality video generation process due to blurry and inconsistent results. In this paper, we introduce a novel generative framework named Dynamic Generative Adversarial Networks (Dynamic GAN) model for regulating the adversarial training and generating photorealistic high-quality sign language videos from skeletal poses. The proposed model comprises three stages of development such as generator network, classification and image quality enhancement and discriminator network. In the generator fold, the model generates samples similar to real images using random noise vectors, the classification of generated samples are carried out using the VGG-19 model and novel techniques are employed for improving the quality of generated samples in the second fold of the model and finally the discriminator networks fold identifies the real or fake samples. Unlike, existing approaches the proposed novel framework produces photo-realistic video quality results without using any animation or avatar approaches. To evaluate the model performance qualitatively and quantitatively, the proposed model has been evaluated using three benchmark datasets that yield plausible results. The datasets are RWTH-PHOENIX-Weather 2014T dataset, and our self-created dataset for Indian Sign Language (ISL-CSLTR), and the UCF-101 Action Recognition dataset. The output samples and performance metrics show the outstanding performance of our model.

An End-to-End Model of ArVi-MoCoGAN and C3D with Attention Unit for Arbitrary-view Dynamic Gesture Recognition

“…In addition, the objective function for training the network is also appropriately designed to measure the similarity in content, action, and view of the generated videos and the real ones. In [28] the authors introduced Dynamic Generative Adversarial Network (Dynamic GAN) model to generate photo-realistic videos from skeletal poses. The proposed model is evaluated on three benchmark datasets of RWTH-PHOENIX-Weather 2014T, Indian Sign Language (ISL-CSLTR), and the UCF-101.…”

Compensation Method for Missing and Misidentified Skeletons in Nursing Care Action Assessment by Improving Spatial Temporal Graph Convolutional Networks

“…It optimized abnormal action weights and provides feedback for skeleton compensation or correction. Building upon the framework of a Generative Adversarial Network [30], we constructed the posture discrimination network and employed this framework to generate regularization loss for pose estimation.…”

Special issue on soft computing for edge-driven applications

“…The third section discusses about the real-time applications of soft computing-driven models by: developing an adaptive neuro-fuzzy inference system to monitor and manage the soil quality in order to build a sustainable farming culture (Remya 2022); introducing a supervised learning-based techniques for predicting the seed germination ability in a precision farming environment (Yasam et al 2022); incorporating an improved sine cosine optimization algorithm for achieving an efficient color image segmentation (Mookiah et al 2022); enhancing singlechannel speech quality and intelligibility under multiple noise conditions by using Wiener filter and deep CNN (Hepsiba and Justin 2021); introducing modified bee algorithm to efficiently classify and predict heart diseases (Velswamy 2021); utilizing the characteristics recognition and soft multimedia system for performing Japanese machine translation and edge-driven hardware implementations (Song 2021); utilizing soft computing-driven social network analysis approach to trace the fake news propagation path (Sivasankari and Vadivu 2021); designing and developing a novel deep learning approach for assisting a gait-based fall prediction model (Murthy et al 2021); and designing and developing a novel generative adversarial networks [GAN] model for generating a high-quality sign language (Natarajan and Elakkiya 2021).…”