The data of a wafer with defects can provide engineers with very important information and clues to improve the yield rate and quality in manufacturing. This paper presents a microscope automatic positioning and wafer detection system with human-machine interface based on image processing and fuzzy inference algorithms. In the proposed system, a XY table is used to move the position of each die on 6 inch or 8 inch wafers. Then, a high-resolution CCD and one set of two-axis optical linear encoder are used to accurately measure the position on the wafer. Finally, the developed human-machine interface is used to display the current position of an actual wafer in order to complete automatic positioning, and a wafer map database can be created. In the process of defect detection, CCD is used for image processing, and during preprocessing, it is required to filter noise, acquire the defect characteristics, define the defective template, and then take the characteristic points of the defective template as the reference input for fuzzy inference. A high-accuracy optical automatic positioning and wafer defect detection system is thus constructed. This study focused on automatic detection of spots, scratches, and bruises, and attempted to reduce the time to detect defective die and improve the accuracy of determining the defects of semiconductor devices.