A highly naturalistic facial expression generation method with embedded vein features based on diffusion model

Abstract

Facial expression generation technology has achieved notable progress in computer vision and artificial intelligence. However, challenges persist regarding background consistency, expression clarity, and detailed representation. Additionally, the instability of generative adversarial networks (GANs) during training affects both image quality and diversity. While diffusion models have demonstrated potential advantages over GANs, research on controllable expression generation remains limited. To address these challenges, this paper proposes a highly natural facial expression generation method based on denoising diffusion implicit models (DDIM) with embedded vein features. This approach avoids adversarial training by employing gradual diffusion to generate specific expressions, thereby enhancing both the diversity and authenticity of the images. Vein features are introduced and embedded within the generated expression images to protect the intellectual property of algorithm-generated digital resources. Firstly, image and expression text guide words are combined as conditional inputs to improve the authenticity and diversity of the generated images. Secondly, a classification coding network is introduced to guide expression generation, thus enhancing the accuracy and consistency of the produced expressions. Furthermore, this paper proposes a vein feature fusion method based on multi-directional local dynamic feature coding operator (MLDFO) and integrates DDIM with frequency-domain watermarking technology to achieve image intellectual property protection. Experimental results demonstrate the effectiveness of this method across several public datasets, including FFHQ, CelebA, FV-USM, and SDUMLA-HMT. Notably, in the CelebA dataset, the average expression recognition rate increased by 11.41%, with a 100.00% recognition rate for happy expressions. The generated expression images exhibit a high degree of authenticity and consistency, and the video conversion tests reveal a natural and smooth effect. These results confirm that this method not only advances facial expression generation technology but also significantly enhances the steganographic protection of images.