Although the current face de-recognition algorithm can effectively protect the user's identity, it will change the original attributes of the image, destroy the visual effect, or change the characteristics of the image to varying degrees. Traditional methods such as pixelation and occlusion can effectively realize face de-recognition, but the processed image cannot be used for basic processing such as face statistics and expression recognition, which greatly destroys the usability of the image in the field of online new media. In order to preserve more attributes of the original image and reduce the changes of facial skin color and texture features caused by the above de-recognition algorithm's operation on the whole facial area, this paper proposes a privacy protection model of face image based on generative adversarial network and facial feature analysis. Our model realizes face recognition based on changing the original image as little as possible. Specifically, we identify and extract features from fixed parts in face images, including eyes, nose, mouth, eyebrows, and cheeks. Then, we designed a face privacy recognition and protection model based on conditional generative adversarial networks. We verify our proposed method on two most commonly used face datasets in face research, namely CelebFaces Attributes and Labeled Faces in the Wild, and the results show that our proposed method achieve better performance compared with other face privacy protection methods.