GAN Lab: Understanding Complex Deep Generative Models using Interactive Visual Experimentation

Abstract

Recent success in deep learning has generated immense interest among practitioners and students, inspiring many to learn about this new technology. While visual and interactive approaches have been successfully developed to help people more easily learn deep learning, most existing tools focus on simpler models. In this work, we present GAN Lab, the first interactive visualization tool designed for non-experts to learn and experiment with Generative Adversarial Networks (GANs), a popular class of complex deep learning models. With GAN Lab, users can interactively train generative models and visualize the dynamic training process's intermediate results. GAN Lab tightly integrates an model overview graph that summarizes GAN's structure, and a layered distributions view that helps users interpret the interplay between submodels. GAN Lab introduces new interactive experimentation features for learning complex deep learning models, such as step-by-step training at multiple levels of abstraction for understanding intricate training dynamics. Implemented using TensorFlow.js, GAN Lab is accessible to anyone via modern web browsers, without the need for installation or specialized hardware, overcoming a major practical challenge in deploying interactive tools for deep learning.

SANVis: Visual Analytics for Understanding Self-Attention Networks

“…Regarding the former, various visual analytic approaches have been proposed for convolutional neural networks mainly computer vision domains [2,6,12,13,19,34] and RNNs in NLP domains [5,11,17,23,24]. Visual analytic approaches have also been integrated with other advanced neural network architectures, such as generative adversarial networks [9,30], deep reinforcement learning [29]. Among them, Strobelt et al [22] developed a visual analytic system for RNN-based attention models, mainly for the exploration and understanding of sequence-to-sequence modeling tasks.…”

Explaining Vulnerabilities to Adversarial Machine Learning through Visual Analytics

“…BOOSTVis [36] and iForest [70] also focus on explaining tree ensemble models through the use of multiple coordinated views to help explain and explore decision paths. Similarly, recent visual analytics work on deep learning [24,25,30,34,44,49,55,[63][64][65]68] tackles the issue of the low interpretability of neural network structures and supports revealing the internal logic of the training and prediction processes.…”

Deep Generative Adversarial Networks for Image-to-Image Translation: A Review

“…The generator takes a random noise vector z (following a Gaussian distribution) as input and outputs a generated sample G(z) without any access to real samples. The discriminator takes both a real sample P data and a generated sample P g as input and predicts the probability of D(x) or D(G(x)) [39,52], as shown in Figure 1.…”

“…GANs have been used to make promising contributions in variety of difficult generative tasks [35], e.g., text-to-photo translation [18], image generation [36], image composition [37], and image-to-image translation [38]. Although GANs are one type of powerful deep generative models, the training of GANs suffers from several issues, such as mode collapse and training instability [39], as discussed in Section 7.1.…”