Semi-automatic tools for nanoscale metrology and annotations for deep learning automation on electron microscopy images

Abstract

For semiconductor applications, billions of objects are manufactured for a single device such as a central processing unit (CPU), storage drive, or graphical processing unit (GPU). To obtain functional devices, each element of the device has to follow precise dimensional and physical specifications at the nanoscale. Generally, the pipeline consists to annotate an object in an image and then take the measurements of the object. Manually annotating images is extremely time-consuming. In this paper, we propose a robust and fast semi-automatic method to annotate an object in a microscopy image. The approach is a deep learning contour-based method able first to detect the object and after finding the contour thanks to a constraint loss function. This constraint follows the physical meaning of electron microscopy images. It improves the quality of boundary detail of the vertices of each object by matching the predicted vertices and most likely the contour. The loss is computed during training for each object using a proximal way of our dataset. The approach was tested on 3 different types of datasets. The experiments showed that our approaches can achieve state-of-the-art performance on several microscopy images dataset.

Deep learning contour-based method for semi-automatic annotation of manufactured objects in electron microscopy images

“…These architectures trained on daily life datasets are not suitable for microscopy images. The inherent challenge is due to the low contrast edges 12 and the image acquisition artifacts. Among the previous cited architectures, E2EC is selected because it was the most promising one on microscopy images based on empirical testing.…”

“…11 In the literature, annotating methods can be broadly classified into three approaches. 12 1.1 Classical Approaches These techniques often involve energy minimization algorithms, like active contours, 13 which deform an initial contour along object boundaries by energy optimization. Other variations, for instance, snakes combined with local shape models, 14 and data-driven energy functions, 15 have been developed.…”