Template matching methods have been widely utilized to detect fabric defects in textile quality control. In this paper, a novel approach is proposed to design a flexible classifier for distinguishing flaws from twill fabrics by statistically learning from the normal fabric texture. Statistical information of natural and normal texture of the fabric can be extracted via collecting and analyzing the gray image. On the basis of this, both judging threshold and template are acquired and updated adaptively in real-time according to the real textures of fabric, which promises more flexibility and universality. The algorithms are experimented with images of fault free and faulty textile samples.