Understanding face matching

Abstract

Many security settings rely on the identity matching of unfamiliar people, which has led this task to be studied extensively in Cognitive Psychology. In these experiments, observers typically decide whether pairs of faces depict one person (an identity match) or two different people (an identity mismatch). The visual similarity of the to-be-compared faces must play a primary role in how observers accurately resolve this task, but the nature of this similarity-accuracy relationship is unclear. The current study investigated the association between accuracy and facial similarity at the level of individual items (Experiment 1 and 2) and facial features (Experiment 3 and 4). All experiments demonstrate a strong link between similarity and matching accuracy, indicating that this forms the basis of identification decisions. At a feature level, however, similarity exhibited distinct relationships with match and mismatch accuracy. In matches, similarity information was generally shared across the features of a face pair under comparison, with greater similarity linked to higher accuracy. Conversely, features within mismatching face pairs exhibited greater variation in similarity information. This indicates that identity matches and mismatches are characterised by different similarity profiles, which present distinct challenges to the cognitive system. We propose that these identification decisions can be resolved through the accumulation of convergent featural information in matches and the evaluation of divergent featural information in mismatches.

A signal detection–based confidence–similarity model of face matching.

“…. This formal aspect is consistent with the idea that each feature contributes an independent share to the overall similarity (see Fysh & Bindemann, 2023). In addition, the variance of the distribution is Var…”

“…As such, it requires observers to take advantage of the visual input from the two photos. The degree of perceived similarity/dissimilarity of a given face pair has been shown to be a good predictor of matching accuracy (Fysh & Bindemann, 2023). Although similarity is considered to be a slippery concept (Tversky, 1977), it is essential for understanding how people recognize and match faces.…”

“…In this process, it is highly likely that the degree of similarity/dissimilarity between the images may play a central role in the quality of matching. Fysh and Bindemann (2023) tested this conjecture by also asking participants to rate the similarity of the face pairs on a 1-6 scale. They found strict relations between accuracy rates and similarity ratings.…”

“…The UVSD can also explain the allegedly puzzling observation that accuracy rates in match and mismatch conditions are not positively correlated at the item level (Fysh & Bindemann, 2023;Megreya & Burton, 2006Wirth & Carbon, 2017). The application of the UVSD to face matching is based on the assumption that observers engage in discriminating between face pairs and not between individual faces.…”

“…For example, an application of the principal component analysis (PCA) to stimuli from the GFMT by Kramer et al (2018) has revealed that the average Euclidean distance for "match" pairs was smaller than for "mismatch" face pairs. The role of similarity in human face matching has been recently investigated by Fysh and Bindemann (2023), who demonstrated that strong lawful relations ensue between accuracy rates and similarity ratings in the face-matching task. In their study, the degree of similarity/dissimilarity between the images in face pairs predicted the pattern of responses to those face pairs both at the global face level and at the featural level.…”