DeepImageJ: A user-friendly environment to run deep learning models in ImageJ

Abstract

DeepImageJ is a user-friendly plugin that enables the generic use in FIJI/ImageJ of pretrained deep learning (DL) models provided by their developers. The plugin acts as a software layer between TensorFlow and FIJI/ImageJ, runs on a standard CPU-based computer and can be used without any DL expertise. Beyond its direct use, we expect DeepImageJ to contribute to the spread and assessment of DL models in life-sciences applications and bioimage informatics.Deep learning (DL) models have a profound impact on a wide range of imaging applications, including life-sciences [1, 2, 3]. Unfortunately, their deployment is often riddled with technical challenges for the non-expert user. Their appropriate use requires insights on sophisticated DL concepts and good programming skills.This situation is in sharp contrast with the philosophy behind ImageJ, the de-facto standard image processing software in life-sciences [4]. This open-source package gives biologists access to a wide variety of user-friendly image analysis tools through third-party plugins and macros.Recent works have aimed at providing a link between TensorFlow 1 and ImageJ 2 . In particular, the CSBDeep team [5], the ImageJ2 team [6], and the Ozcan Research Group [7,8] have pioneered this connection by making their pre-trained TensorFlow models accessible through ImageJ. Unfortunately, this connection effort has remained restricted to their specific applications.We present DeepImageJ 3 , an open-source plugin of ImageJ that runs a variety of third-party TensorFlow models in a generic way. Keras models can also be integrated whenever properly converted to TensorFlow's format SavedModels. DeepImageJ is designed as a standard ImageJ plugin with the technicalities hidden behind a user-friendly interface.

The ImageJ ecosystem: Open‐source software for image visualization, processing, and analysis

“…DeepImageJ (https://deepimagej.github.io/deepimagej/) is a plugin that enables access to deep learning models from within Fiji or ImageJ (Figure 6). 36 Users need no experience with deep learning but can simply download available deep learning models from a central repository and embed them directly into ImageJ or Fiji 36 . The execution of the pretrained deep learning model makes use of the Java bindings of TensorFlow 36 .…”

“… 36 Users need no experience with deep learning but can simply download available deep learning models from a central repository and embed them directly into ImageJ or Fiji 36 . The execution of the pretrained deep learning model makes use of the Java bindings of TensorFlow 36 . Since many models require certain pre‐ and postprocessing of the image data, DeepImageJ also hosts and serves those in the form of ImageJ macros or scripts.…”

HistoFlow: Label-Efficient and Interactive Deep Learning Cell Analysis

“…The ‘Trainable Weka Segmentation’ plugin (13) for ImageJ (14) and its distribution Fiji allows to train and apply machine learning, but focuses on shallow learning. The newer DeepImageJ plugin (5) allows to apply deep learning models and features ready-to-use models for a variety of tasks such as segmentation, super-resolution and virtual staining. However, it does not support to train them or execute them on a remote GPU server.…”

“…While many tools allow the application of neural networks to microscopy images and some even provide a simple to use model repository for it (4) (5) (6) (7), to our knowledge, there is currently no tool that enables users to train a deep neural network in the same integrated environment.…”

DeepMIB: User-friendly and open-source software for training of deep learning network for biological image segmentation

“…Unfortunately, in many cases, the application of these methods is not easy and requires significant knowledge in computer sciences, making it difficult to adapt by many researchers. Software developers have already started to address this challenge by developing userfriendly deep learning tools, such as Cell Profiler [10], Ilastik [11], ImageJ plug-ins DeepImageJ [12] and Unet [13], CDeep3M [5], and Uni-EM [14] that are especially suitable for biological projects. However, the overall usability is limited because they either rely on pre-trained networks without the possibility of training on new data [10][11][12], are limited to electron microscopy (EM) datasets [14], or have specialized computing requirements [5,13].…”