The Feasibility of External Blind DNA Proficiency Testing. I. Background and Findings

Abstract

We describe the origins, purposes, and findings of a national study to determine whether a large-scale program of blind proficiency testing in U.S. DNA laboratories is feasible and/or practical. Proficiency testing in clinical laboratories is reviewed, particularly as mandated by the Clinical Laboratory Improvement Acts and its role in the regulation of those laboratories. Proficiency testing in forensic urine drug testing labs is also briefly reviewed. Studies involving comparisons between open and blind proficiency testing are discussed. The clinical laboratory proficiency testing and regulation landscape provides the background for the DNA Act of 1994, and the congressional mandate to investigate blind proficiency testing in forensic DNA laboratories. Four models of blind proficiency testing are defined and discussed, along with the advantages and disadvantages of each and estimates of the costs of a large-scale program. The purposes of proficiency testing in a quality-assurance context are likewise discussed and related to the models and the arguments generally proffered for and against blind vs. open proficiency testing.

An experimental study of blind proficiency tests in forensic science

“…Presumably, however, the intended meaning was that it would cost $500,000 to $1 million to give all of the assumed 150 DNA labs in the system one test. This interpretation puts the NAS statement close to the correct value found in Peterson et al (2003a). This statement still exaggerates the cost estimated in the study.…”

“…Their "Estimates Extrapolated from This Project" range from $2,000 per test to $3,500 per test. The value imputed to Peterson et al (2003a) by the NAS report might seem to be about two orders of magnitude larger. Presumably, however, the intended meaning was that it would cost $500,000 to $1 million to give all of the assumed 150 DNA labs in the system one test.…”

“…This statement may be easily misunderstood, however, and it seems to contain an error. Peterson et al (2003a) provide a table summarizing their cost estimates. Their "Estimates Extrapolated from This Project" range from $2,000 per test to $3,500 per test.…”

Legal perceptions of forensic DNA profiling

“…Similar views have been raised when surveys into competency levels of forensic scientists suffered from restricted access to data [41]. In one study to assess the feasibility of external blind proficiency testing overseen by the US National Institute of Justice, only 39 DNA testing facilities out of the 151 approached agreed to voluntarily participate [42]. This perception is therefore understandable, although not easily explained.…”

Using organizational economics to engage cultural key masters in creating change in forensic science administration to minimize bias and errors

“…Blind proficiency tests have been criticized as being expensive Peterson et al (2003aPeterson et al ( , 2003b. conducted a proof of principle to determine whether blind proficiency tests are feasible.…”