Faulty science?
The opening sentence in the news report is ominous.
According to a by-lined article by Spencer S. Hsu in the Washington Post, “The FBI has notified crime labs across the country that it has discovered errors in data used by forensic scientists in thousands of cases to calculate the chances that DNA found at a crime scene matches a particular person.”1
The article goes on to explain that the issue is with the population statistics part of its DNA analysis. The FBI currently uses 13 specific DNA markers to identify individuals as the likely source of a DNA sample; it’s in the process of transitioning to using 20 markers.
Nobody is arguing about whether the results of a specific test are correct in terms of the markers: if the test says 12 match, then 12 match. The issue is what that 12-marker match means.
That’s where population statistics come into play. That’s the part that the forensic scientists then use to say how likely it is that two different people would both have the same markers.
Now as the scientist who initially noted the problem and reported it to the FBI says, in many cases, the outcome of a case isn’t likely to change based on the issue he identified. There really isn’t much difference between a one-in-six-billion chance and a one-in-one-billion chance of two random individuals sharing the same markers.
But things change — and they change dramatically — when the DNA sample isn’t a good one and the odds that are given are a lot lower. The scientist, Wright State University statistics professor Daniel R. Krane, was quoted by the Washington Post as saying he’d been involved in a case where the odds were given as a 1-in-180 chance of matching. If you cut that number in half — or by an even greater percentage — then its evidentiary impact is dramatically changed.
Unlike the tempest-in-a-teapot over the recent unsuccessful attempt by police to use genetic genealogy markers to help identify a murder suspect,2 this really is a big deal. Real people have faced real charges and real consequences based on what may turn out to be, in their particular cases, faulty science.
The fact that only a small percentage of outcomes in cases where DNA has been used may have been changed by the faulty analysis isn’t any comfort at all. Our justice system is based on the notion that we’d rather 10 guilty men go free than one innocent man go to jail.
The upshot of this disclosure by the FBI is that every defense attorney in the country who’s handled a case involving DNA since 1999 is going to take another look at the evidence used in that case. Challenges will be filed in trial and appeals courts across the country trying to overturn convictions where the jury could possibly have been misled by the population statistics part of DNA evidence.
With some luck, and a lot of hard work by the defense community, some injustices will be corrected.
And we can hope that the police — including the FBI — will have learned a lesson. Because — the story says — this was reported a decade ago… and the FBI downplayed it then. At a time when it could have been corrected. And the consequences limited, if not eliminated.
We also need the courts to be better gatekeepers when it comes to scientific evidence. We all know how much people rely on this sort of evidence, and judges need to be sure it isn’t being oversold when it’s used in court.
Those are the takeaways for the justice system.
So what’s the takeaway for genetic genealogy here?
One more time: it’s that we need to be crystal clear, whenever we’re talking about genetic genealogy tests, that the kinds of tests we do are inappropriate for use by the police when they’re trying to advance their investigations.
The tests we do are designed to show how people are alike, how they share characteristics that make them part of one family.
The tests the police do are designed to show how people are unique, how each individual is different from everyone else on the face of the earth.
Clearly, the police should be very very wary of ever trying to use the kinds of tests we do to help in any way in their investigations. They have enough trouble with their own tests, intended to identify one — and only one — specific person who has committed a crime, without adding to the confusion by using our tests, intended to identify shared ancestry.
SOURCES
- Spencer S. Hsu, “FBI notifies crime labs of errors used in DNA match calculations since 1999,” The Washington Post, online edition, posted 29 May 2015 (http://www.washingtonpost.com : accessed 30 May 2015). ↩
- See Justin Petrone, “Ancestry.com Shutters SMGF Database Amid Murder Case Controversy,” GenomeWeb, posted 28 May 2015 (https://www.genomeweb.com/ : accessed 30 May 2015). ↩
I remember when this came out, and everyone seemed to just look the other way. Add faulty analysis to sometimes sloppy sample collection and/or processing, and perhaps some faulty assumptions as well, and that’s a setup for the kind of errors in justice that we dread. It calls into question even good procedures. While no case is decided on the basis of DNA alone, the number of wrongful convictions where DNA isn’t even involved should give us pause. I think your point is well taken about the difference in genealogical DNA sampling and forensic DNA. The science is stronger, for one thing, and the process cleaner. And the outcome is altogether different, as you so well point out.
Been a lot of people convicted on just DNA evidence alone. This could be a nightmare for the justice system and a blessing for the innocent. The downside is even the guilty might go free, but in our system of justice, its beyond a reasonable doubt, we must abide by that regardless!
That’s dreadful. (On the plus side, it’s nice to know that faculty from the university where I teach are thinking about the statistical problem.)
We need more data on the nature of the 33 errors in the PopStats reference database before we can extrapolate to wrongful convictions. According to the news article, details will be published shortly in a technical journal. I expect the changes will cause minor differences in the allele frequencies of individual markers, which will not have a substantive impact on random match probabilities when taken in combination for all 13 markers. Krane’s hypothetical example must involve a different scenario entirely. In any event, I’m sure defense attorneys will be scrutinizing this in great detail.