A color pattern recognition scheme with encoding technique based on the nonzero order joint transform correlator is proposed. Each color target image is organized as a new grayscale encoded image. A Lagrange multipliers approach is also used to design an optimized synthetic reference function. A joint transform power spectrum subtraction method is applied to remove the zero order terms in addition. Simulation results verify the performance of the proposed technique. This correlator can reduce the channel number in the input and still maintain good ability in color image recognition. © 2007 Wiley Periodicals, Inc. Microwave Opt Technol Lett 49: 1665–1669, 2007; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/mop.22554