Deep learning‐based automatic inpainting for material microscopic images

Abstract

Summary The microscopic image is important data for recording the microstructure information of materials. Researchers usually use image‐processing algorithms to extract material features from that and then characterise the material microstructure. However, the microscopic images obtained by a microscope often have random damaged regions, which will cause the loss of information and thus inevitably influence the accuracy of microstructural characterisation, even lead to a wrong result. To handle this problem, we provide a deep learning‐based fully automatic method for detecting and inpainting damaged regions in material microscopic images, which can automatically inpaint damaged regions with different positions and shapes, as well as we also use a data augmentation method to improve the performance of inpainting model. We evaluate our method on Al–La alloy microscopic images, which indicates that our method can achieve promising performance on inpainted and material microstructure characterisation results compared to other image inpainting software for both accuracy and time consumption. Lay Description A basic goal of materials data analysis is to extract useful information from materials datasets that can in turn be used to establish connections along the composition–processing–structure–properties chain. The microscopic images obtained by a microscope is the key carrier of material microstructural information. Researchers usually use image analysis algorithms to extract regions of interest or useful features from microscopic images, aiming to analyse material microstructure, organ tissues or device quality etc. Therefore, the integrity and clarity of the microscopic image are the most important attributes for image feature extraction. Scientists and engineers have been trying to develop various technologies to obtain perfect microscopic images. However, in practice, some extrinsic defects are often introduced during the preparation and/or shooting processes, and the elimination of these defects often requires mass efforts and cost, or even is impossible at present. Take the microstructure image of metallic material for example, samples prepared to microstructure characterisation often need to go through several steps such as cutting, grinding with sandpaper, polishing, etching, and cleaning. During the grinding and polishing process, defects such as scratches could be introduced. During the etching and cleaning process, some defects such as rust caused by substandard etching, stains etc. may arise and be persisted. These defects can be treated as damaged regions with nonfixed positions, different sizes, and random shapes, resulting in the loss of information, which seriously affects subsequent visual observation and microstructural feature extraction. To handle this problem, we provide a deep learning‐based fully automatic method for detecting and inpainting damaged regions in material microscopic images, which can automatically inpaint damaged regions with different positions and shapes, as well as we also us...

Medical Image Recognition Technology in the Effect of Substituting Soybean Meal for Fish Meal on the Diversity of Intestinal Microflora in Channa argus

“…Healthy gastrointestinal tract is very important for overall health, and animal health depends on the intake of nutrients, the balance of digestive juice (acid and enzyme), and the important role of microorganisms in the digestive system [ 20 ]. The research on promoting microecology in aquatic animals is still in its infancy, but its importance has been recognized and related research has been carried out [ 21 , 22 ]. Facing the current situation of rapid development and large scale of China's animal husbandry and feed industry, in order to reduce the dependence of China's feed industry on fish meal and other expensive animal protein sources, it is very important to develop cheaper and easily available protein products to meet the animal's needs for feed protein nutrition [ 23 , 24 ].…”

Toward developing tangling noise removal and blind inpainting mechanism based on total variation in image processing

“…Proposed tangling noise removal mechanism: Similar to the representation in [11], we rewrite the damaged forward process (1) as the new formula,…”

“…When solving each subproblem, the other variables are fixed as constant by the last iteration results. Therefore, by alternating iteration, we have the following iteration scheme to solve (11): where (18) is the Chambolle's projection algorithm for solving TVl 2 problem (12), (19), and (21) are the results of Newton's method for solving (13) and (15), and (20) is the hard thresholding for solvingl 0 − l 2 problem (13).…”

“…When solving each subproblem, the other variables are fixed as constant by the results of the last iteration. Therefore, by alternating iteration, we have the following iteration scheme to solve (11). When solving each subproblem, the other variables are fixed as constant by the last iteration results.…”

“…Blind painting is more realistic because, for most corrupted images, the area of missing pixels is not known in advance. Several researchers used various deep learning approaches for image inpainting [10,11]. In [12], Dong et al proposed a tight frame model for blind inpainting and removing additive Gaussian white noise.…”