Automatic fabric defect detection has been successfully applied to establish the quality quick response system for the automation of textile production. However, the image complexity and diversity of patterned fabrics have effects on the fabric defect detection, which makes it difficult for automatic quality inspection of textiles. In order to solve this problem, a novel method that sequential detection of image defects for patterned fabrics was proposed in this paper. Firstly, the fabric image was segmented adaptively based on the periodic distance of the pattern, which can be used for the determination of repetitive pattern block element in fabrics, so that the defect image blocks could be identified according to the minimum principle of structure similarity; Then, the texture features of defect image blocks were extracted to match the constructed feature dictionary of defect-free image block, the defect position could be located by distance measurement and threshold segmentation. Our experiments show that the proposed method has the advantages of low computational complexity, high detection accuracy and strong applicability, by the comparison with existing approaches for defect detection of patterned fabric. Moreover, the new proposed automatic defect location method can solve the problem of deficiency caused by manual defect marking using existing image inpainting algorithms, it can be used for the automatic image inpainting of patterned fabrics without manual marking to improve the robustness of computer vision-based fabric re-engineering. INDEX TERMS patterned fabric, quality inspection, defect detection, defect position, image inpainting.