We report on the first controlled study comparing the abilities of forensic document examiners (FDEs) and laypersons in the area of signature examination. Layersons and professional FDEs were given the same signature-authentication/simulation-detection task. They compared six known signatures generated by the same person with six unknown signatures. No a priori knowledge of the distribution of genuine and nongenuine signatures in the unknown signature set was available to test-takers. Three different monetary incentive schemes were implemented to motivate the laypersons. We provide two major findings: (i) the data provided by FDEs and by laypersons in our tests were significantly different (namely, the hypothesis that there is no difference between the assessments provided by FDEs and laypersons about genuineness and nongenuineness of signatures was rejected); and (ii) the error rates exhibited by the FDEs were much smaller than those of the laypersons. In addition, we found no statistically significant differences between the data sets obtained from laypersons who received different monetary incentives. The most pronounced differences in error rates appeared when nongenuine signatures were declared authentic (Type I error) and when authentic signatures were declared nongenuine (Type II error). Type I error was made by FDEs in 0.49% of the cases, but laypersons made it in 6.47% of the cases. Type II error was made by FDEs in 7.05% of the cases, but laypersons made it in 26.1% of the cases.

Reliable data on the capabilities of professional document examiners are scarce, rendering most past characterizations of these capabilities somewhat speculative. We report on a comprehensive test administered to more than 100 professional document examiners, intended to close this data gap in the area of writer identification. Each examiner made 144 pair-wise comparisons of freely-created original handwritten documents. The task was to determine whether or not a "match" was detected, namely whether or not the two documents were written by the same hand. Matching criteria were based on the identification and strong probability definitions of the ASTM standard E1658. The professionals were tested in three groups (in the northeastern, southeastern, and southwestern United States). In addition, we have created a control group of similar educational background. Several individuals training to become professional document examiners were tested as well.Examination of the data and statistical tests show that the answers collected from the professional and nonprofessional groups came from different populations. The trainees' data were shown to have come from a population that is distinct from both professional and nonprofessional groups. Unlike the professional examiners, the nonprofessionals tended to grossly over-associate. They erroneously "matched" many documents that were created by different writers, mismatching almost six times as many unknown documents to database documents as the professionals did (38.3% vs. 6.5% of the documents).The results of our test lay to rest the debate over whether or not professional document examiners possess writer-identification skills absent in the general population. They do.

A comprehensive writer identification test was designed and administered to a group of professional document examiners and to a control group of nonprofessionals. The professional group consisted of seven document examiners from the Federal Bureau of Investigation. The control group consisted of ten graduate students in the areas of engineering and business. The main finding is that the professional document examiners performed significantly better than the members of the control group. The hypothesis that professionals and nonprofessionals are equally proficient in performing writer identification was found via the Kruskal-Wallis test to have probability of less than 0.001. These findings give indication that handwriting identification expertise indeed exists, and calls into question the conclusions of previous studies in this area.

No abstract

In September 1997 we reported on a comprehensive proficiency test administered to three groups of professional document examiners (105 individuals). Each test-taker performed 144 pairwise comparisons of original handwritten documents and matched together pairs that in his/her opinion were generated by the same hand. The test was also administered to a control group of nonprofessionals (41 individuals) whose educational profile was similar to that of the tested professionals. These nonprofessionals were motivated through a monetary incentive plan ($25 gain for each correct match: $25 fine for each erroneous match; $10 fine for failure to match documents created by the same hand). In this paper, we report on a subsequent study, aimed to discover whether changes in the monetary incentive scheme would affect the performance of nonprofessionals, and whether these schemes would close the performance gap between professionals and nonprofessionals. We administered the 1997 test again, this time to four groups of nonprofessionals (132 subjects), using four different incentive schemes (including the one used originally). We found that the four sets of data obtained under different incentives were indistinguishable, in the sense that differences between the test scores were not statistically significant. We conclude that the performance of nonprofessionals in our proficiency test was relatively insensitive to the monetary incentive scheme.