Textile pattern recognition and recoloring separates a fabric image into different color regions and changes colors of the regions in the fabric image. A new method for textile pattern recognition and recoloring is proposed. The proposed method consists of two main steps. First, the binary images are recognized through a series of image enhancement techniques, such as the color‐encoding algorithm, the edge‐based filtering algorithm, and a multiphase image segmentation model. Second, the regions of interest are recolored with different recoloring methods based on intrinsic image decomposition and grayscale image recoloring. Experimental results and comparisons demonstrate that the main disadvantages and difficulties of using palette‐based methods in textile pattern recoloring have been overcome by the proposed method. The proposed method is effective in inhibiting tiny region effects and preserving the geometry structures of textile patterns. It can be applied to recognize and recolor different kinds of fabric images, including the Jacquard, the embroidered, and the printed fabric images.