Evaluating the Interpretability of Generative Models by Interactive Reconstruction

GANSlider: How Users Control Generative Models for Images using Multiple Sliders with and without Feedforward Information

“…Recent research in Human Computer Interaction (HCI) and AI used generative models interactively as well (e.g. see [12,24,30] and Figure 2), including work on co-creativity and computational creativity (e.g. [27,28]).…”

“…We used an image reconstruction task (cf. Figure 1) similar to Ross et al [30]. We measured all interaction events plus subjective feedback.…”

“…An increasing number of control dimensions corresponds to a larger latent image space that can be explored, and thus adds to the complexity of interactive tasks in that space. Currently, there is no guidance on the number of control dimensions to use in various settings, with numbers reported in related work ranging from 5 to 80 dimensions [12,22,30]. Second, we hypothesize that visual feedforward information in the UI may aid users in their decisionmaking when working with generative models (i.e.…”

“…In contrast, such disentangled dimensions are not always readily available to (AI) researchers while developing new generative models. As a new way to handle this, researchers have worked on methods to explore and evaluate generative models via interaction [12,30]. This often involves visualizing a model's learned dimensions to judge their interpretability and quality: A popular method in (AI) research publications is to plot multiple images generated at different points along said dimensions (e.g.…”

“…Today, most user interfaces (UIs) for generative models use a set of regular sliders, in which each one is mapped to the control of one latent dimension. Such UIs have been used to enable interaction for model exploration and evaluation [12,22,30]. However, slider UIs themselves have not been at the focus of any empirical study in this context so far.…”