Impact of Digital Face Beautification in Biometrics

Abstract

Face retouching is a widespread procedure available across a huge spectrum of modern applications. Among them, social media offer different filters to beautify face pictures by performing operations such as skin smoothing, addition of virtual makeup, as well as deforming certain facial features, for instance by widening the eyes or making the nose thinner. In this work, the effect of different facial feature modification filters (FFMF) on face recognition (FR), gender classifiers and a weight estimator are studied. To this end, popular FFMF are applied to face images of the publicly available CALFW and VIP attribute databases. Such filters distort or modify biometric features, affecting the ability of automatic FR systems to recognize individuals. The results show that the application of FFMF to face images penalizes the accuracy of FR systems and affects the estimation of other facial traits such as gender and weight.

A Case Study on how Beautification Filters Can Fool Deepfake Detectors

“…Inside those, an area of study is focused on evaluating the potential harm that the addition of artificial elements, such as flower crowns or puppy ears, to face images via social media filters might pose to face recognition systems [1], [12]. More in line with our investigation, other works assessed the impact that beautification filters have on society [23] and in the estimation of different biometrics traits namely face, gender and weight [21].…”

“…Social media platforms offer a diverse range of tools referred to as "filters" designed to automatically enhance a user's image, demanding minimal or no user proficiency [21]. Certain types of filters are designed to tweak different facial features such as skin, lips, eyes, and nose to enhance the beauty of the user.…”

“…Some common modifications are makeup addition, narrow noses, skin tanning and smoothening. Beautification filters have been demonstrated as a disturbance factor for AI facial processing tasks such as face recognition and gender classification [21]. Despite deepfake detection technology being challenged against several video processing operations [20], its robustness against social media beautification effects has not yet been tested.…”

Mirror, Mirror on the Wall, Who Is the Whitest of All? Racial Biases in Social Media Beauty Filters

“…However, the beautification process only involved the superimposition of simple AR elements that create occlusions on the face. Mirabet Herranz et al (2022) studied the impact of beauty filters on both face recognition and esti mation of biometric features by beautifying the CALWF (T. Zheng et al, 2017) and VIP_attribute (Dantcheva et al, 2018) data sets. Riccio et al (2022) proposed openfilter, a framework to apply any filter available on social media platforms to existing collections of faces, sharing the beautified version of two large and publicly available face data sets (fairbeauty and b-lfw).…”

“…However, the beautification process only involved the superimposition of simple AR elements that create occlusions on the face. Mirabet Herranz et al (2022) studied the impact of beauty filters on both face recognition and estimation of biometric features by beautifying the CALWF (T. Zheng et al, 2017) and VIP_attribute (Dantcheva et al, 2018) data sets.…”