Critical examination of the IREX VI results

Abstract

The authors analyse why Iris Exchange Report (IREX) VI conclusions about 'iris ageing' differ significantly from results of previous research on 'iris template ageing'. They observe that IREX VI uses a definition of 'iris ageing' that is restricted to a subset of International Organization for Standardization (ISO)-definition template ageing. They also explain how IREX VI commits various methodological errors in obtaining what it calls its 'best estimate of iris recognition ageing'. The OPS-XING dataset that IREX VI analyses for its 'best estimate of iris recognition ageing' contains no matches with Hamming distance >0.27. A 'truncated regression' technique should be used to analyse such a dataset, which IREX VI fails to do so, biasing its 'best estimate' to be lower-than-correct. IREX VI mixes Hamming distances from first, second and third attempts together in its regression, creating another source of bias towards a lower-than-correct value. In addition, the match scores in the OPS-XING dataset are generated from a '1-to-first' matching strategy, meaning that they contain a small but unknown number of impostor matches, constituting another source of bias towards an artificially low value for ageing. Finally, IREX VI makes its 'best estimate of iris recognition ageing' by interpreting its regression model without taking into account the correlation among independent variables. This is another source of bias towards an artificially low value for ageing. Importantly, the IREX VI report does not acknowledge the existence of any of these sources of bias. They conclude with suggestions for a revised, improved IREX VI.

Analysis of the effect of ageing, age, and other factors on iris recognition performance using NEXUS scores dataset

“…To address these concerns, NIST undertook an effort to better understand the effect of ageing and other factors on iris biometrics [2]. This effort opened a whole new range of questions related to the factors that affect iris recognition and the ways iris biometrics is evaluated [4][5][6][7][8][9].…”

“…These metrics are listed in Table 1. The metrics that were shared with NIST and UND and used in the previous research [2][3][4][5][6][7][8][9] are marked in bold.…”

“…The OPS‐XING dataset, a part of which was used in the IREX VI evaluation by NIST [2, 4] and the evaluations conducted by UND [5, 6, 9], consists of over a quarter billion of matching and IQ metrics that were recorded during enrolment and passage transactions by the NEXUS system. These metrics are listed in Table 1.…”

“…In 2012, a portion of anonymised NEXUS kiosk log data was shared with the National Institute of Standard and Technology (NIST) scientists for the IREX VI study aimed at investigating the effect of aging on iris performance [2], where it was labelled the OPS-XING dataset. The results of IREX VI study, which were published in 2013, brought a lot of attention from the scientific community, actively discussed and contested by the scientists from the University of Notre Dame [3][4][5][6][7][8][9].…”

Interpretable Deep Learning-Based Forensic Iris Segmentation and Recognition

“…One of the strengths of iris as a biometric identifier is its high temporal stability compared to other biometric modes [22]. Although the assessment of how biological changes impact the systems' decisions (not only similarity scores) remains still one of the research challenges awaiting an authoritative answer [8,23,32,14,39], observations from operational deployments suggest that iris recognition may be a good candidate for a "cradle-to-grave" biometrics across all human identification approaches. However, one of the recent discoveries made by several research groups was that iris recognition is possible to be done also after death [7,40].…”