SAR: Scale-Aware Restoration Learning for 3D Tumor Segmentation

A Point in the Right Direction: Vector Prediction for Spatially-aware Self-supervised Volumetric Representation Learning

“…This is a desirable property for tasks with many biomedical objects with extremely limited labels. Our outperformance over other methods that also use reconstruction [9][10][11] shows both the efficacy of our spatial pretext task and superior data efficiency. 3) Both our encoder and decoder are initialized with multi-scale and semantically diverse tasks.…”

“…By encapsulating the three key spatial properties into an interdependent task, we prevent training instability and feature interference from independent predictions like when classifications were used [10]. Additionally, vectors represent spatial properties in a continuous space which is a generalized extension of discrete rotation or scale prediction [13].…”

“…However, many 3D medical tasks have prohibitively small datasets while requiring cubically more memory, which is incongruous with contrastive methods' reliance on large datasets, batch sizes, and models [7]. 2) Reconstruction [2,[8][9][10][11] has been shown to outperform supervised transfer learning on 3D data by restoring an original image from a noised or masked version. Still, when analyzing reconstruction results in [9] (see the left side of Fig.…”

“…This may suggest an over-focus on local intensity statistics over more robust semantic or spatial understandings of the whole structures. 3) Prediction-based pretext tasks entail solving spatial "puzzles" such as relative position prediction [1], jigsaw unscrambling [1,12,13], or scale classification [10]. However, these methods are prone to short-cut learning and over-emphasize discriminative elements (e.g., patch boundaries).…”

“…However, these methods are prone to short-cut learning and over-emphasize discriminative elements (e.g., patch boundaries). Moreover, some approaches propose to learn spatial properties incompletely [10] or indirectly through disparate pretext tasks [9,14], but implicit learning is suboptimal and uncontrolled. To our best knowledge, no approach adequately covers all key spatial properties important to 3D medical image analysis in practice.…”