GANs for Biological Image Synthesis

Abstract

In this paper, we propose a novel application of Generative Adversarial Networks (GAN) to the synthesis of cells imaged by fluorescence microscopy. Compared to natural images, cells tend to have a simpler and more geometric global structure that facilitates image generation. However, the correlation between the spatial pattern of different fluorescent proteins reflects important biological functions, and synthesized images have to capture these relationships to be relevant for biological applications. We adapt GANs to the task at hand and propose new models with casual dependencies between image channels that can generate multichannel images, which would be impossible to obtain experimentally. We evaluate our approach using two independent techniques and compare it against sensible baselines. Finally, we demonstrate that by interpolating across the latent space we can mimic the known changes in protein localization that occur through time during the cell cycle, allowing us to predict temporal evolution from static images.

Building a 3D Integrated Cell

“…Since the post of our initial preprint manuscript and software (Johnson et al, 2017), there has been increased interest in the application of GANs to modeling cell morphology and organization. (Osokin et al, 2017) constructed a network that can generate images of cell shape, and generates additional subcellular structures conditioned on those. (Goldsborough et al, 2017) uses a GAN to generate images of cells across different conditions and evaluate the performance of a discriminator in identifying the mechanism of action of these conditions.…”

“…(Goldsborough et al, 2017) uses a GAN to generate images of cells across different conditions and evaluate the performance of a discriminator in identifying the mechanism of action of these conditions. These other GAN approaches (Osokin et al, 2017;Goldsborough et al, 2017), allow neither a probabilistic interpretation nor prediction of structure localization, in real images. Furthermore, the methods presented above generate 2D images with one to two orders of magnitude more training data than the method presented here.…”

When Deep Learning Meets Cell Image Synthesis

“…The simplest way is to generate the synthetic image data from the real one using some type of geometrical transform (linear or nonlinear), which is often used in data augmentation techniques. Furthermore, it is possible to generate the whole synthetic image after training without distinguishing any regions using generative adversarial networks (GANs) but this approach requires a large amount of training data, works slowly even in 2D (no paper in 3D so far) and does not offer inherent ground truth, which makes it impractical for both benchmarking and data augmentation purposes.…”

CytoGAN: Generative Modeling of Cell Images

“…Pawlowski [15] first investigated autoencoder-based methods but reported results far inferior to hand-tuned features or transfer learning approaches. Our model is more related to the work by Osokin et al [14] and Johnson et al [8], wherein GANs were used to model cell images, although their applications did not include morphological profiling.…”

“…GANs have been proven to synthesize realistic images both within and beyond the biological domain [7,17,14]. We examine if this ability transfers to cell images extracted from the BBBC021 dataset of human breast cancer cell lines.…”