FRVT 2006 and ICE 2006 large-scale results

Abstract

This report describes the large-scale experimental results from the Face Recognition Vendor Test (FRVT) 2006 and the Iris Challenge Evaluation (ICE) 2006. The FRVT 2006 looks at recognition from high-resolution still images and three-dimensional (3D) face images, and measures performance for still images taken under controlled and uncontrolled illumination. The ICE 2006 reports iris recognition performance from left and right iris images. The FRVT 2006 results from controlled still images and 3D images document an order-of-magnitude improvement in recognition performance over the FRVT 2002. This order-of-magnitude improvement was one of the goals of the preceding technology development effort, the Face Recognition Grand Challenge (FRGC). The FRVT 2006 and the ICE 2006 compared recognition performance from very-high resolution still face images, 3D face images, and single-iris images. On the FRVT 2006 and the ICE 2006 datasets, recognition performance was comparable for all three biometrics. In an experiment comparing human and algorithm performance, the best-performing face recognition algorithms were more accurate than humans. These and other results are discussed in detail.

Stable face representations

“…This is true even under highly restricted testing conditions, based on tightly controlled, high resolution images and cooperative subjects. Under these optimal conditions, accuracy levels as high as 99 per cent have been achieved [17,18]. The problem is that when conditions are not so favourable (as in border control or surveillance settings), or cooperation is poor (for example, when someone is trying to conceal his or her identity), performance plummets.…”

“…How are we to account for the discrepancy between the rather impressive performance of automatic face recognition systems on benchmark tests [17,18] and their unusable performance in the real world? Evidently, one of these situations does not capture the applied problem.…”

“Are Machines Better Than Humans in Image Tagging?” - A User Study Adds to the Puzzle

“…Phillips et al [17] conducted one of the first comparisons of face identification capabilities of humans and machines. Interestingly, at that time the top three algorithms were already able to match or to do even better face identification compared with human performance on unfamiliar faces under illumination changes.…”

3D Face Recognition: Technology and Applications

“…In the presence of literally hundreds of alternative systems, independent benchmarks are needed to evaluate alternative algorithms and to assess the viability of 3D face against other biometric modalities such as high-resolution 2D Faces, fingerprints and iris scans. Face Recognition Grand Challenge (FRGC) [76] and Face Recognition Vendor Test 2006 (FRVT'06) [73] are the two important evaluations where the 3D face modality is present. Face Recognition Grand Challenge: FRGC is the first evaluation campaign that focuses expressly on face: 2D Face at different resolutions and illumination conditions and 3D face, alone or in combination with 2D [76,75].…”

“…The FRVT 2006 is an independent large-scale evaluation campaign that aims to look at performance of high resolution 2D and 3D modalities [73] together with other modalities.…”