Earlier this year, authors Leila Schneps and Coralie Colmez published their book Math on Trial.
In it, they discuss a number of cases where, they allege, mathematical errors were made in court. One of the cases discussed is the case against Amanda Knox and Raffaele Sollecito.
The authors explain that, in this case, the Appeal Court's decision to not re-test the DNA evidence found on the so-called double-DNA knife was flawed. In an article in the New York Times, they summed up their position as follows:
"One of the major pieces of evidence was a knife collected from Mr. Sollecito’s apartment, which according to a forensic scientist contained a tiny trace of DNA from the victim. Even though the identification of the DNA sample with Ms. Kercher seemed clear, there was too little genetic material to obtain a fully reliable result — at least back in 2007.
By the time Ms. Knox’s appeal was decided in 2011, however, techniques had advanced sufficiently to make a retest of the knife possible, and the prosecution asked the judge to have one done. But he refused. His reasoning? If the scientific community recognizes that a test on so small a sample cannot establish identity beyond a reasonable doubt, he explained, then neither could a second test on an even smaller sample.
Whatever concerns the judge might have had regarding the reliability of DNA tests, he demonstrated a clear mathematical fallacy: assuming that repeating the test could tell us nothing about the reliability of the original results. In fact, doing a test twice and obtaining the same result would tell us something about the likely accuracy of the first result. Getting the same result after a third test would give yet more credence to the original finding."
Is this criticism fair? Did the Appeal Court flunk its math exam?
Well, to my mind the answer to this is yes and no. The Appeal Court ultimately did err in its assessment of probabilities, but, ironically, it did not err in the way the Schneps and Colmez seem to think.
Why is this? Well, for a very simply reason: the authors misrepresent the Appeal Court's reasoning. After all, when it comes to the question of whether a new test should be preformed, this is what the Appeal Court actually said:
"In fact, (the prosecution) argued that systems currently exist able to analyse such low quantities, albeit still at a developmental stage. This Court holds, however, that it is precisely the fact they are still under development, in practice in an experimental phase, which precludes us from basing a belief in guilt on the results obtained with the application of such systems: the Judge can do no else but base his or her opinions on the technical systems and established scientific knowledge from a particular time period – the period in which s/he is called to judge – and not on others still in an experimental phase. This, once again, to reach a decision of guilty beyond any reasonable doubt."
So what does this mean? Well, the Appeal Court is definitely not saying "the first test yielded an inconclusive result; the second test would yield another inclusive result, so let's not do the new test". What the court is saying, is that any new test (regardless of the outcome) would be irrelevant, simply because such a new test would have to be carried out by "systems" that are "still in development" and "experimental" and, therefore, inherently untrustworthy.
Schneps and Colmez seem to think that this a case where you could have had two single results, both of which might be quite acceptable, but both of which, when considered singularly, are inconclusive. They seem to think that the Appeal Court made the basic error of not considering that two such results might well prove to be much more relevant when taken together. However, that is clearly not what the Appeal Court has done.
So much, then, for Schneps's and Colmez's argument. They have simply misrepresented the Appeal Court's reasoning and, based on that misrepresentation, erroneously assumed the court made some sort of mathematical error.
The question remains, though - did the Appeal Court do its math properly? I would say not.
Well, Schneps and Colmez are right in one thing. The Appeal Court does seem to have muddled its understanding of the law of probabilities. However, what's in question is not the probability of just one or two DNA tests leading to a reliable result with regard to the knife; instead, it's the probability of the entire case. That is: of all the bits and pieces that, when fitted together, drew the original court to its conclusion that that Knox and Sollecito must be guilty.
What the Appeal Court has done is that it looks at all these bits and pieces separately. It then rejects them all. Not because they could not possibly lead to the conclusion that Knox and Sollecito are guilty, but rather because, seen singularly, they do not lead to that inevitable conclusion. And, the Appeal Court then reasons that, since there is not one single bit of evidence that would, in itself, prove their guilt beyond a reasonable doubt, all the various bits and pieces taken together wouldn't either.
Now this is a clear error, and it can be simply demonstrated.
Let's look at this from a simple mathematical point of view and assume that there are various aspects of the case which could point to guilt and which might not, and let's assume that each aspect has a 50% to 50% ratio between the two.
For example: the break-in. There's a 50% chance it was real; there's a 50% chance it was staged. Kercher's DNA on the knife? A 50% chance it was there; a 50% chance it wasn't.
Now let's sum up a number of the most important factors, two of which I've just mentioned. Here's a somewhat simplified list:
Break-in 50% 50%
DNA knife 50% 50%
DNA bra clasp 50% 50%
Luminol traces 50% 50%
Footprint on mat 50% 50%
DNA traces bathroom 50% 50%
Total 50% 50%
From this simple list, anyone would assume that there's a 50% chance that Knox and Sollecito are guilty, and a 50% chance that they are innocent. So that would clearly implicate that they should be cleared in court; the Appeal Court correctly acquitted them, right?
Well, not so. The thing is, of all these various factors, there is really only one that must be taken into consideration when assuming guilt. That's the break-in. As I've stated earlier, there is no way in which Knox and Sollecito might be guilty if the break-in actually occurred. To put it another way, if the apartment was actually broken into, one must assume their innocence.
Such an assumption need not in any way be made when it comes to any of the other factors, however. The assumption that Kercher's DNA was on not the knife, for example, does not in any way lead to conclusion that Knox and Sollecito must be innocent. The same applies to the DNA found on the bra clasp; the same applies to the Luminol traces, etc.
Conversely, if any of these factors did indeed conclusively point to Knox's or Sollecito's involvement, any single factor would be sufficient to establish their guilt. If, for example, it must be assumed that Sollecito's footprint was found on the mat in the small bathroom, it must be assumed that Knox and Sollecito are indeed guilty.
So let's do the math. Are Knox and Sollecito guilty? Well, there's a 50% startling chance. Does that get any lower? No; none of the factors mentioned in my list can decrease that. Can it get any higher? Certainly: if the chances of their innocence decrease, the chances of their guilt rise proportionally.
Are they innocent? Well, again you start with a 50% chance. Does that get any lower? Oh, yes. You start out with the break-in, which is where your initial 50% comes from. But next you'd have to assume that Kercher's DNA is not on the knife. So that's another 50%. You're left with 50% x 50% = 25%. Then you move to the bra clasp. Another 50%. That makes 12,5%. You move on through the remaining factors, and end up with a rather stunning 0,78% chance of them being innocent. Yes, that's right: less than 1%. Conversely, there's a higher than 99% chance that they're guilty.
Now, don't get me wrong. I'm not seriously suggesting that you could settle the whole case by simply doing a few sums. What I am suggesting, though, is that there is a basic error in the Appeal Court's reasoning.
The error is very simple. You can't look at this case and say that there's not a single piece of evidence that necessarily leads to Knox's and Sollecito's guilt, and then leave it at that. You have to look at all the pieces of evidence, and you have to look at all those pieces together. If you do, the picture becomes quite different, and it simply becomes rather difficult to assume their innocence.
Can that assumption still be made? Yes, I would say it can. Just not in the way in which the Appeal Court has attempted it. Its logic isn't very sound, and neither is its math.