Automated inspection of semiconductor defect data has become increasingly important over the past several years as a means of quickly understanding and controlling contamination sources and process faults, which impact product yield. To address the issue of too much data and too little time, automation technologies in defect detection and review are being developed by universities, laboratories, industry, and semiconductor equipment suppliers. In this thesis, a new rule-based approach is proposed to segment defect images. Several segmentation techniques already exist but they often focus on the constraints of a specific application and therefore they lack of generality and flexibility. This limits the use of computer vision in all those tasks where the visual data content and the purpose of the defect analysis are not known a priori. Moreover, the limited generality increases the costs for the design of unsupervised defect image analysis systems. To overcome these limitations, it is proposed to decompose a general defect image segmentation problem in four levels. The lowest levels have a high degree of generality and are inspired by the perceptual mechanisms of human vision. Their main role is to simplify the input data and to extract the perceptually meaningful information. The proposed process is sufficiently general to be applied to a wide class of applications and input data without the need of human supervision. In order to achieve a higher degree of autonomy, generality and a more flexible solution, the proposed defect image segmentation approach adopts a hierarchical perspective. The visual information in a frame is viewed as being composed of patterns, referred to as objects, which are built from simpler sub-patterns, referred to as regions, which are themselves built from yet simpler primitives corresponding to the pixels in the defect image. vi