DeepProjection: specific and robust projection of curved 2D tissue sheets from 3D microscopy using deep learning

Abstract

The efficient extraction of image data from curved tissue sheets embedded in volumetric imaging data remains a serious and unsolved problem in quantitative studies of embryogenesis. Here we present DeepProjection (DP), a trainable projection algorithm based on deep learning. This algorithm is trained on user-generated training data to locally classify the 3D stack content and rapidly and robustly predict binary masks containing the target content, e.g., tissue boundaries, while masking highly fluorescent out-of-plane artifacts. A projection of the masked 3D stack then yields background-free 2D images with undistorted fluorescence intensity values. The binary masks can further be applied to other fluorescent channels or to extract the local tissue curvature. DP is designed as a first processing step than can be followed, for example, by segmentation to track cell fate. We apply DP to follow the dynamic movements of 2D-tissue sheets during dorsal closure in Drosophila embryos and of the periderm layer in the elongating Danio embryo. DeepProjection is available as fully documented Python package.

Bellybutton: accessible and customizable deep-learning image segmentation

“…The chosen network architecture strikes a balance between being small enough to train rapidly from scratch on a laptop, while being large enough to generate valid segmentation on nontrivial problems. The choice of a CNN has been the standard for segmentation problems 6 , 12 , 14 , 18 , 20 , 22 – 26 , as it allows the network natural access to spatial information. The decreasing layer size is also standard, and gives the network sufficient flexibility to hierarchically analyze spatial patterns without superfluous parameters.…”

“…The chosen network architecture strikes a balance between being small enough to train rapidly from scratch on a laptop, while being large enough to generate valid segmentation on nontrivial problems. The choice of a CNN has been the standard for segmentation problems 6,12,14,18,20,[22][23][24][25][26] , as it allows the network natural access For training, a user may provide individually-labeled segmentation maps, that is, every pixel in a particular segment must contain the same number, unique to that segment. Alternatively, if no segments are in contact, a user-provided binary mask is sufficient.…”

“…As a result, a large and recent body of work has been focused on methods and software packages for simplifying this process. The majority focused on biological research, specifically the tracking of cells from microscopy data [9][10][11][12][13][14][15][16] , but similar works tackle goals ranging from identifying and tracking 2D materials like graphene 17 to segmenting other medical or biological imaging data [18][19][20][21] , images of flora and fauna 22 , scanning electron microscopy images for material science 23,24 , astronomical data 25,26 , particle physics 27 , and more. Typically these works compete for highest accuracy on benchmark data sets 11 , or ease of use for pre-specified domains (very often biological data) 9,10 .…”

Minimal vertex model explains how the amnioserosa avoids fluidization duringDrosophiladorsal closure

“…Two-dimensional projections of the AS tissue were created from 3D stacks using DeepProjection (70). A custom Python algorithm was used to segment and track individual AS cells throughout dorsal closure (32): Briefly, binary masks of the AS cell boundaries and the amnioserosa tissue boundary (leading edge) were first predicted from microscopy movies using deep learning trained with expert-annotated dorsal closure specific data (71).…”

Mechano-chemical regulation of complex cell shape formation: Epidermal pavement cells—A case study

“…Other approaches use projection methods where the three-dimensional shape of the cells is considered. This is done either by creating entire 3D models that are computationally more complex or by creating a surface mesh that represents the original cell shape as a curved plane, often referred to as a 2.5D model (Barbier de Reuille et al, 2015 ; Eng et al, 2021 ; Erguvan et al, 2019 ; Haertter et al, 2022 ; Herbert et al, 2021 ; Schneider et al, 2022 ).…”