BIAFLOWS: A Collaborative Framework to Reproducibly Deploy and Benchmark Bioimage Analysis Workflows

Abstract

Summary Image analysis is key to extracting quantitative information from scientific microscopy images, but the methods involved are now often so refined that they can no longer be unambiguously described by written protocols. We introduce BIAFLOWS, an open-source web tool enabling to reproducibly deploy and benchmark bioimage analysis workflows coming from any software ecosystem. A curated instance of BIAFLOWS populated with 34 image analysis workflows and 15 microscopy image datasets recapitulating common bioimage analysis problems is available online. The workflows can be launched and assessed remotely by comparing their performance visually and according to standard benchmark metrics. We illustrated these features by comparing seven nuclei segmentation workflows, including deep-learning methods. BIAFLOWS enables to benchmark and share bioimage analysis workflows, hence safeguarding research results and promoting high-quality standards in image analysis. The platform is thoroughly documented and ready to gather annotated microscopy datasets and workflows contributed by the bioimaging community.

Bioimage analysis workflows: community resources to navigate through a complex ecosystem

“…But this is now changing as image processing tools and visual feedback are now getting incorporated in SWMS. 21,31,33 Towards findability and accessibility of image analysis workflows At a general level in life-science and not specifically for the bioimage analysis community, coordination efforts are ongoing in the direction of the "FAIRification" of workflows, but also the ease to access HPC resources to run them. They are led by European Research Infrastructures, such as ELIXIR.…”

“…BIAFLOWS 21 is an open-source web platform to reproducibly deploy and publicly benchmark image analysis workflows with a strong focus on microscopy bioimages. The database stores scientific datasets, metadata, and versioned image analysis workflows with parameters optimized for the corresponding datasets.…”

“…Table 3 shows examples of workflow management systems with usage in bioimage analysis. This containerization can be performed at the level of each individual workflow component (such as in Galaxy 30 , 31 ), or for complete workflows (such as in BIAFLOWS 21 or coming to grand-challenges ). Biocontainers 32 is proposing a standard and recipes for these containerizations, as well as a marketplace for the containers, today mostly for –omics data processing.…”

“…But this is now changing as image processing tools and visual feedback are now getting incorporated in SWMS. 21 , 31 , 33 …”

SSNOMBACTER: A collection of scattering-type scanning near-field optical microscopy and atomic force microscopy images of bacterial cells

“…While such tools have already been reported for AFM imaging, they are still largely unavailable for the more recent s-SNOM modality, whose spread and number of applications have escalated over the past years [ 12 ]. The importance of reference datasets that enable objective comparisons between competing microscopy-oriented image analysis/processing approaches is discussed in detail by Rubens et al [ 74 ]. In the following, we discuss potential use-cases of our dataset.…”