Forensic science is an applied science. It involves testing, examination, analysis, interpretation. And it can be erroneous.
What comprises an “error” in scientific testing? For example, if the test or procedure involves identifying a drug or substance, and the result is “inconclusive,” is that considered an error? Someone has to make that decision. (Most people would say it wasn’t.) The point is, the discussion of error rates is not as clear-cut as one might think.
An error rate can be thought of as: the number of times a result from a test or procedure is wrong, compared to the number of times the test or procedure has been done.
Error can be expressed as a percentage, as a decimal number, or as a numerical fraction, e.g. 1/N (one in N times).
But a number of factors muddy the process of determining an error rate.
Perhaps you want to find the error rate of a particular test. How complicated is that test? How often is it done? The more complicated the test, the less frequently it may be done; and the less frequently it’s done, the harder it is to measure the error rate. Given variations in measurements, possibly due to human error or degree of precision, it’s important to note that uncertainty measurement, differs from measurement error, which is making conclusions about the measurement.
Error rates are considered for several reasons in the context of reliability. For one, they help us assess whether the problematic forensic issues in a case lie with the discipline as a whole, with the local laboratory, with a particular technique, or with the specific analyst.
Once an error rate has been established, another challenge awaits: what is considered an acceptable error rate? Suppose there were a procedure with an error rate that all scientists could agree on as 1 out of 10,000. Would a judge disallow the testimony of an analyst who’d used that procedure? If so, would there be an error rate the judge could deem acceptable?
The Daubert decision regarding admissibility doesn’t contemplate an arbitrary cutoff. The more significant point is that if we know the error rate, the trier can make a more intelligent decision as to how much weight to ascribe to the test result.
Are we better off with a test that has an unknown rate, or one with a known, substantial rate?
Not all errors are the same. That is to say, they don’t all have the same impact. The American Society of Crime Lab Directors, Lab Accreditation Board defines three “classes” of errors: