Self-supervised training for blind multi-frame video denoising

Dewil, Valéry; Anger, Jérémy; Davy, Axel; Ehret, Thibaud; Facciolo, Gabriele; Arias, Pablo

doi:10.1109/wacv48630.2021.00277

Cited by 25 publications

(18 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the computer vision literature, related to natural images, the usefulness of pre-training has been widely explored for several tasks. Namely, the coloring of a grayscale image [50,51,52], the restoration of a distorted or deteriorated image [53,54,55,56], the prediction of the transformation performed in an image [57] or even, the re-ordering of pieces or frames of images [58,59] and videos [60]. However, there is hardly any work applying this methodology to microscopy images.…”

Section: Related Workmentioning

confidence: 99%

Deep learning based domain adaptation for mitochondria segmentation on EM volumes

Franco-Barranco¹,

Pastor-Tronch²,

Gonzalez-Marfil³

et al. 2022

Preprint

View full text Add to dashboard Cite

Background and Objective: Accurate segmentation of electron microscopy (EM) volumes of the brain is essential to characterize neuronal structures at a cell or organelle level. While supervised deep learning methods have led to major breakthroughs in that direction during the past years, they usually require large amounts of annotated data to be trained, and perform poorly on other data acquired under similar experimental and imaging conditions. This is a problem known as domain adaptation, since models that learned from a sample distribution (or source domain) struggle to maintain their performance on samples extracted from a different distribution or target domain. In this work, we address the complex case of deep learning based domain adaptation for mitochondria segmentation across EM datasets from different tissues and species. Methods: We present three unsupervised domain adaptation strategies to improve mitochondria segmentation in the target domain based on (1) state-ofthe-art style transfer between images of both domains; (2) self-supervised learning to pre-train a model using unlabeled source and target images, and then fine-tune it only with the source labels; and (3) multi-task neural network architectures trained end-to-end with both labeled and unlabeled images. Additionally, to ensure good generalization in our models, we propose a new training stopping criterion based on morphological priors obtained exclusively in the source domain. The code and its documentation are publicly available at https://github.com/danifranco/EM_domain_adaptation Results: We carried out all possible cross-dataset experiments using three publicly available EM datasets. We evaluated our proposed strategies and those of others based on the mitochondria semantic labels predicted on the target datasets. Conclusions: The methods introduced here outperform the baseline methods and compare favorably to the state of the art. In the absence of validation labels,

show abstract

Section: Related Workmentioning

confidence: 99%

Deep learning based domain adaptation for mitochondria segmentation on EM volumes

Franco-Barranco¹,

Pastor-Tronch²,

Gonzalez-Marfil³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The codes used for our experiments are based on Pytorch, and we have used the official code 8 for RAFT [43]. Our method introduces three additional hyper parameters, namely, (λ 1 , λ 2 ) of Eq.10 and of Eq.8.…”

Section: Implementation Detailsmentioning

confidence: 99%

“…between two consecutive frames in a video [9,19,40,43]. It is widely employed in video analysis applications including video compression [32,46], action recognition [6,29], video denoising [3,8], and object tracking [25,52], to point out a few.…”

Section: Introductionmentioning

confidence: 99%

Imposing Consistency for Optical Flow Estimation

Jeong¹,

Lin²,

Porikli³

et al. 2022

Preprint

View full text Add to dashboard Cite

Imposing consistency through proxy tasks has been shown to enhance data-driven learning and enable selfsupervision in various tasks. This paper introduces novel and effective consistency strategies for optical flow estimation, a problem where labels from real-world data are very challenging to derive. More specifically, we propose occlusion consistency and zero forcing in the forms of selfsupervised learning and transformation consistency in the form of semi-supervised learning. We apply these consistency techniques in a way that the network model learns to describe pixel-level motions better while requiring no additional annotations. We demonstrate that our consistency strategies applied to a strong baseline network model using the original datasets and labels provide further improvements, attaining the state-of-the-art results on the KITTI-2015 scene flow benchmark in the non-stereo category. Our method achieves the best foreground accuracy (4.33% in Fl-all) over both the stereo and non-stereo categories, even though using only monocular image inputs.

show abstract

“…Our test set is composed from clips downloaded from 14 noncompressed clips of the Derf's Collection [6], that is commonly used to evaluate video coding and streaming [13], super resolution [15], compression artifact reduction [11] and visual quality improvement tasks [4]. The clips were compressed from 1080p to 540p with the H.265 codec with CRF 23, preparing an analogue setting as the train set.…”

Section: Datasetmentioning

confidence: 99%