Cut and Learn for Unsupervised Object Detection and Instance Segmentation

Wang, Xudong; Girdhar, Rohit; Yu, Stella X.; Misra, Ishan

doi:10.48550/arxiv.2301.11320

Cited by 4 publications

(3 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These approaches are typically fully supervised. Then, unsupervised models have been also developed such as Cut and Learn, 29 DINO, 30 and LOST 31 which unfortunately have displayed lower performances with respect to their supervised counter parts. Both deep approaches take time for training but are more robust during inference.…”

Section: Deep Learning Approachesmentioning

confidence: 99%

Deep learning contour-based method for semi-automatic annotation of manufactured objects in electron microscopy images

Sanou,

Baderot,

Bricq

et al. 2024

J. Electron. Imag.

View full text Add to dashboard Cite

Precision characterization is fundamental to achieve expected performance in semiconductors where Moore's law pushes the boundaries to miniaturize components. To measure these attributes, deep learning models are used, which require manual annotation of several objects captured via electron microscopy. However, this annotation can be laborious and time-consuming. We propose a semi-automated method for annotating items in electron microscopy images, in an effort to be innovative, efficient, and reliable. Our approach involves identifying objects, enhancing boundaries with use of a unique loss function incorporating physical aspects from electron microscopy images. It greatly reduces the need for users to undertake the annotation model's training process. It also minimizes post-inference processing by delivering a ready-to-use model. The constrained dynamic match loss (C-DML) incorporates dynamic matching with horizontal/vertical symmetry constraints to address the distinct challenges presented by manufactured objects acquired by microscopy imaging. Metrology metrics from the contour predictions obtained with C-DML obtain a mean relative error (MRE) of <10% and a correlation coefficient surpassing 90% when compared with ground truth corresponding to manual annotations. Our experimental results demonstrate the superior performance of C-DML over both classical DML and state-of-the-art deep annotation models. An extensive investigation demonstrates the effectiveness of our approach on heterogeneous datasets, including diverse objects of different materials and shapes, leading to state-of-the-art measurement results. Additionally, we show with the experiments that we can obtain better performances with better hyperparameters and data augmentation. Furthermore, this investigation presents a technique for annotating electron microscopy images efficiently and sheds light on the essential elements that dictate the approach's overall efficacy.

show abstract

Section: Deep Learning Approachesmentioning

confidence: 99%

Deep learning contour-based method for semi-automatic annotation of manufactured objects in electron microscopy images

Sanou,

Baderot,

Bricq

et al. 2024

J. Electron. Imag.

View full text Add to dashboard Cite

show abstract

“…Another line of work seeks object-centric scene decomposition without reconstruction. This includes works such as [61,72,28,70,3,48]. However, unlike ours, these approaches lack the ability to generate images.…”

Section: Clevrtexmentioning

confidence: 99%

Object-Centric Slot Diffusion

Jiang¹,

Deng²,

Singh³

et al. 2023

Preprint

View full text Add to dashboard Cite

Despite remarkable recent advances, making object-centric learning work for complex natural scenes remains the main challenge. The recent success of adopting the transformer-based image generative model in object-centric learning suggests that having a highly expressive image generator is crucial for dealing with complex scenes. In this paper, inspired by this observation, we aim to answer the following question: can we benefit from the other pillar of modern deep generative models, i.e., the diffusion models, for object-centric learning and what are the pros and cons of such a model? To this end, we propose a new object-centric learning model, Latent Slot Diffusion (LSD). LSD can be seen from two perspectives. From the perspective of object-centric learning, it replaces the conventional slot decoders with a latent diffusion model conditioned on the object slots. Conversely, from the perspective of diffusion models, it is the first unsupervised compositional conditional diffusion model which, unlike traditional diffusion models, does not require supervised annotation such as a text description to learn to compose. In experiments on various object-centric tasks, including the FFHQ dataset for the first time in this line of research, we demonstrate that LSD significantly outperforms the state-ofthe-art transformer-based decoder, particularly when the scene is more complex. We also show a superior quality in unsupervised compositional generation.

show abstract

“…Some of these methods can be applied to different types of images which are not present in the training set. We can mention also some models trained in an unsupervised way like Cut and Learn for Unsupervised Object Detection and Instance Segmentation, 16 DINO, 17 LOST. 18 Recently, some models of these approaches combine deep learning especially, CNN for feature extractions and the use of a classical approach like snake algorithm in order to find the contour.…”

Section: Introductionmentioning

confidence: 99%

Semi-automatic tools for nanoscale metrology and annotations for deep learning automation on electron microscopy images

Sanou¹,

Baderot²,

Benezeth³

et al. 2023

Sixteenth International Conference on Quality Control by Artificial Vision

View full text Add to dashboard Cite

For semiconductor applications, billions of objects are manufactured for a single device such as a central processing unit (CPU), storage drive, or graphical processing unit (GPU). To obtain functional devices, each element of the device has to follow precise dimensional and physical specifications at the nanoscale. Generally, the pipeline consists to annotate an object in an image and then take the measurements of the object. Manually annotating images is extremely time-consuming. In this paper, we propose a robust and fast semi-automatic method to annotate an object in a microscopy image. The approach is a deep learning contour-based method able first to detect the object and after finding the contour thanks to a constraint loss function. This constraint follows the physical meaning of electron microscopy images. It improves the quality of boundary detail of the vertices of each object by matching the predicted vertices and most likely the contour. The loss is computed during training for each object using a proximal way of our dataset. The approach was tested on 3 different types of datasets. The experiments showed that our approaches can achieve state-of-the-art performance on several microscopy images dataset.

show abstract

Cut and Learn for Unsupervised Object Detection and Instance Segmentation

Cited by 4 publications

References 36 publications

Deep learning contour-based method for semi-automatic annotation of manufactured objects in electron microscopy images

Deep learning contour-based method for semi-automatic annotation of manufactured objects in electron microscopy images

Object-Centric Slot Diffusion

Semi-automatic tools for nanoscale metrology and annotations for deep learning automation on electron microscopy images

Contact Info

Product

Resources

About