This paper presents a novel method to search face candidate regions in color images. First, a skin-color model is used to get the skin regions. Then, particle swarm optimization (PSO) is utilized for searching face candidate regions, which can save time and eliminate small noises. Experimental results show that this searching method is super to the conventional method which scans the whole image pixel by pixel. Finally, BP neural network is used to verify the face. The output error formula is modified to make the neural network converge more quickly. Bootstrap method is utilized to choose the training samples for the network, which reduces the correlation between samples and improves the detection effects. This approach is robust and can achieve high detection rate when detecting frontal faces, a little rotated and profile faces. In addition, it can detect faces with different size, expression, as well as having glasses and beard.