Tests of face processing are typically designed to identify individuals performing outside of the typical range; either prosopagnosic individuals who exhibit poor face processing ability, or super recognisers, who have superior face processing abilities. Here we describe the development of the Oxford Face Matching Test (OFMT), designed to identify individual differences in face processing across the full range of performance, from prosopagnosia, through the range of typical performance, to super recognisers. Such a test requires items of varying difficulty, but establishing difficulty is problematic when particular populations (e.g. prosopagnosics, individuals with Autism Spectrum Disorder) may use atypical strategies to process faces. If item difficulty is calibrated on neurotypical individuals, then the test may be poorly calibrated for atypical groups, and vice versa. To obtain items of varying difficulty we used facial recognition algorithms to obtain face pair similarity ratings that are not biased towards specific populations. These face pairs were used as stimuli in the OFMT, and participants required to judge if the face images depicted the same individual or different individuals. Across five studies the OFMT was shown to be sensitive to individual differences in the typical population, and in groups of both prosopagnosic individuals and super recognisers. The test-retest reliability of the task was at least equivalent to the Cambridge Face Memory Test and the Glasgow Face Matching Test. Furthermore, results reveal, at least at the group level, that both face perception and face memory are poor in those with prosopagnosia, and good in super recognisers.