Take our Survey Bookmark and Share
Examining the Evidence

Understanding Error

Weighing the Evidence > Reliability > Understanding Error
Forensic science is an applied science. It involves testing, examination, analysis, interpretation. And it can be erroneous.

Defining Error Rates

What comprises an “error” in scientific testing? For example, if the test or procedure involves identifying a drug or substance, and the result is “inconclusive,” is that considered an error? Someone has to make that decision. (Most people would say it wasn’t.) The point is, the discussion of error rates is not as clear-cut as one might think.

An error rate can be thought of as: the number of times a result from a test or procedure is wrong, compared to the number of times the test or procedure has been done.

Error can be expressed as a percentage, as a decimal number, or as a numerical fraction, e.g. 1/N (one in N times).

But a number of factors muddy the process of determining an error rate.

Perhaps you want to find the error rate of a particular test. How complicated is that test? How often is it done? The more complicated the test, the less frequently it may be done; and the less frequently it’s done, the harder it is to measure the error rate. Given variations in measurements, possibly due to human error or degree of precision, it’s important to note that uncertainty measurement, differs from measurement error, which is making conclusions about the measurement.

Error rates are considered for several reasons in the context of reliability. For one, they help us assess whether the problematic forensic issues in a case lie with the discipline as a whole, with the local laboratory, with a particular technique, or with the specific analyst.

Acceptable Error Rates

Once an error rate has been established, another challenge awaits: what is considered an acceptable error rate? Suppose there were a procedure with an error rate that all scientists could agree on as 1 out of 10,000. Would a judge disallow the testimony of an analyst who’d used that procedure? If so, would there be an error rate the judge could deem acceptable?

The Daubert decision regarding admissibility doesn’t contemplate an arbitrary cutoff. The more significant point is that if we know the error rate, the trier can make a more intelligent decision as to how much weight to ascribe to the test result.

Are we better off with a test that has an unknown rate, or one with a known, substantial rate?

Classification of Error

Not all errors are the same. That is to say, they don’t all have the same impact. The American Society of Crime Lab Directors, Lab Accreditation Board defines three “classes” of errors:

  • Class 3: An error determined to have a minimal effect, that is unlikely to recur, and does not affect the fundamental reliability of the laboratory’s work
  • Class 2: An error that is more serious than Class 3, but not persistent enough to cause immediate concern over the lab’s overall work
  • Class 1: A serious error. The nature and cause of the error call into question the reliability of the laboratory’s work.