Computer Vision Techniques Applied to the Reconstruction of the 3-D Structure Dislocations

Abstract

Computer-vision, nowadays well integrated into electron microscopy sciences, is becoming an essential implement for the advancement of this field, in particular to take the most advantage of the cutting-edge electron microscopes. Computer vision techniques can be applied for automated detection and segmentation of structures in 2-and 3-D micrographs and 3-D reconstruction, as well as for deriving quantitative data from electron microscopy data [1][2][3].3-D reconstruction of one-dimensional crystal defects called dislocations reveals key information about their network geometry and dominant interaction mechanisms [4]. Conventional tomography techniques in transmission electron microscopy (TEM) usually use a tilt-series of projections to retrieve the 3-D structure of many objects, including dislocation lines, through different reconstruction schemes [5]. The linear shape of dislocations can be incorporated as an important prior knowledge to lower the number of images that are needed for a reliable reconstruction.In this contribution, we integrate several computational technologies including machine learning, segmentation methodologies, epipolar geometry, and triangulation to develop an efficient method for multi-view 3-D reconstruction of dislocation lines. We use state-of-the-art convolutional neural networks, specifically a U-Net, to first delineate dislocations and subsequently finding corresponding points as optical flow [6]. 3-D reconstruction is performed using an affine camera model estimated from point correspondences.The strength of this method is experimentally demonstrated on the reconstruction of dislocations using bright-field, weak-beam and high-angle annular dark-field S/TEM images. As demonstrated in figure 1, dislocation lines were accurately traced in the 2-D BF-STEM images of a heteroepitaxial gallium nitride (GaN) membrane. Our 3-D reconstruction algorithm employed well-established computer vision techniques to match contours across images and to infer their 3-D shape. Finally, the 3-D configuration of these dislocations was reconstructed using these two images whose viewing directions are separated by only 3.2° [7]. The algorithm also takes the specificity of the setup into account by introducing customized smoothing techniques based on Gaussian filters of different sizes to overcome the nonhomogeneous nature of the noise along different axes. The experiments that have been performed on real and synthetic data assert the approach is able to reach a significant precision in the 3-D reconstruction of dislocations lines even using a single pair of stereo images with a stereo tilt angle that spans from a few to some tens of degrees [7][8][9].