A new high-speed foreign fiber detection system with machine vision is proposed for removing foreign fibers from raw cotton using optimal hardware components and appropriate algorithms designing. Starting from a specialized lens of 3-charged couple device (CCD) camera, the system applied digital signal processor (DSP) and field-programmable gate array (FPGA) on image acquisition and processing illuminated by ultraviolet light, so as to identify transparent objects such as polyethylene and polypropylene fabric from cotton tuft flow by virtue of the fluorescent effect, until all foreign fibers that have been blown away safely by compressed air quality can be achieved. An image segmentation algorithm based on fast wavelet transform is proposed to identify block-like foreign fibers, and an improved canny detector is also developed to segment wire-like foreign fibers from raw cotton. The procedure naturally provides color image segmentation method with region growing algorithm for better adaptability. Experiments on a variety of images show that the proposed algorithms can effectively segment foreign fibers from test images under various circumstances.