Most defect inspection methods used in semiconductor manufacturing require design layout or golden die images.Unlike methods that require such additional information, this paper presents a method for automatic inspection of defects in semiconductor images with a single image.First, we devise a method to classify images into four types: flat, linear, patterned, and complex using a cosine similarity.For linear and patterned images, we obtain defect-free images that retain the structure.Then, subtract defect-free image from input image to get a flat image.The FAST-MCD method then estimates the parameters of the inlier distribution of the flat image and uses them to detect defects.A segmentation neural network is used to detect defects in complex images.