Surprising Probability (or why we learn more from our mistakes, than from our successes).

Simple statistics show that I learn more from my mistakes, yet I only get rewarded for success. blog.mindrocketnow.com

http://en.wikipedia.org/wiki/Bayes'_theorem#Drug_testing

Suppose a test for Mind Rockets is 99% sensitive and 99% specific. That is, the test will produce 99% true positive results for Mind Rockets and 99% true negative results for non-Mind Rockets. Suppose that only 0.5% of people are actually Mind Rockets. If a randomly selected individual tests positive, what is the probability he or she is a Mind Rocket?

Despite the apparent accuracy of the test, if an individual tests positive, it is twice as likely that they are not actually positive, than they are. This surprising result arises because the number of non-Mind Rockets is very large compared to the number of Mind Rockets, so a small percentage of a large number is still larger than a large percentage of a small number.

To use concrete numbers: if 1000 individuals are tested, there are expected to be 995 non-Mind Rockets and 5 Mind Rockets. From the 995 non-Mind Rockets, 9.95 false positives are expected. From the 5 Mind Rockets, 4.95 true positives are expected. So in our sample of 1000, out of 14.9 expected positive results, only 4.95 (33.2%) are genuine.

If we flip this around, out of the 995 non-Mind Rockets, we expect to find 985.05 true negatives. And in the 5 Mind Rockets, we expect to find 0.05 false negatives. Therefore out of the sample of 1000, we can expect to find 985.1 negatives, of which 99.99% are genuine. So we learn more from the negative results than from the positive results.

Why is this important? As The Economist pointed out, academic research, particularly pharmaceutical research, disproportionately rewards positive results, despite negative results (and even replicating results) being more meaningful. Moreover, society in general rewards success and doesn’t acknowledge the positives in failure.

I can see that this is changing in the school where I am a Governor; we’re trying very hard to “scaffold” the learning so that the children learn through failure as well as success. But after that, further schooling and then work is still all about high marks in SATs and performance-based bonuses, and is at odds with personal growth.

We need our education system to be more entrepreneurial. I don’t mean that we need all of our students to be dot com millionaires or winners of the Dyson prize. We need our education system to give our children the confidence to treat both imposters of triumph and disaster the same. Because that’s the only way that they will be able to fill each unforgiving minute with sixty seconds worth of distance run.