Notis 237, 2012-04-06. Diskussion och frågor om Bayesian statistik, frågor från Jan Dalkvist, svar från Jessica Utts, ledande parapsykolog och professor i statistik. Hur kan man ta hänsyn till den tro/misstro man har till ESP-hypotesen? Hur ska man kvantifiera tro? Kan kvantifierad tro likstämmas med objektiv sannolikhet?  

/Jan Dalkvist + GB

Resonemang av Jan Dalkvist:

Jag kan tänka mig att en del av er som har försökt följa Bem-debatten har blivit förvirrade. Jag tänker på den Bayesianska statistiken, där man inför så kallade a priori- sannolikheter, t.ex. för hypotesen att ESP existerar. Några sådana sannolikheter kan man självklart inte räkna ut i samma mening som man kan räkna ut sannollikheten att råka ut för en bilolycka om man bor i en storstad. Vad man i själva verket gör är att sätta en siffra på sin tro på att ESP existerar givet vissa fakta, t.ex. att Ganzfeld-experimenten på det hela taget har gett positiva resultat. Det är här det hela blir lite konstigt och subjektivt. T.ex., en forskare som tror att de positiva GF-resultaten beror på fusk och slarv kommer att bedöma sannolikheten som låg att resultaten stöder ESP, medan en forskare som anser att GF-experimenten är välgjorda bedömer sannolikheten som hög. Inte undra på att forskare som förlitar sig på Bayesiansk statistik kan komma till radikalt olika slutsatser. Man skulle, med en viss överdrift, kunna kalla den Bayesianska statistiken för en slags ”gummistatistik”, som tillåter var och en att komma fram till de slutsatser som han eller hon önskar uppnå. Grundfelet: man likställer objektiv sannolikhet med kvantifierad tro.

Frågan vi/jag då å Jan Ds vägnar ställde till Jessica Utts var:

With the Daryl Bem-study, some of us, knowing statistics very well, have been a little confused how to use the Bayesian statistics. So I would like to shortly share with you these confused ideas.

If I understand it right, so-called a-priori-propabilities are introduced, for e g the hypothesis that ESP exists. And as I understand it, such probabilities can not be calculated in the same way as for having a traffic accident living in a big city.

What you do? marking your belief on a scale given certain facts, e g that the Ganzfeld-experiments on the whole have given positive results.

And I here I/we guess it can be a little strange and subjective. A scientist believing that the positive GF-results depend on cheating and being careless will judge the probability to be low that the results will support ESp, while a researcher believing that these experiments are well done, will judge the probability to be high.

With this idea, it is very easy to come to quite different conclusions with Bayesian statistics?

You can with some exaggeration call the Bayesian statistics for a kind of ”rubber statistic”, allowing anybody to draw the conclusions that he or she wants to reach?

How can you come around the basic wrong step to equalize objective probability with quantified belief?

Och hennes svar:

It is correct that people with different prior probabilities will come to different conclusions. But also there are two different ways in which prior opinion comes into the analysis.
1. Prior belief that the null hypothesis is true (no ESP). 2. Prior belief about how large the effect is, if ESP is real.
The 2nd one is harder to specify. This is where the criticism of Daryl’s paper by Wagenmakers et al went wrong. The prior belief they used for #2 was too big. They said if ESP is real, it is a large effect. We all know that isn’t true. So, in their analysis, the choice was between:
No ESP A large effect ESP
Of course the data supported the ”No ESP” answer, given those two choices.
Daryl and I, with my colleague Wes Johnson, wrote a response to their analysis, which explains all this. I’m attaching the paper.
One reason I like the Bayesian approach is that everyone can be honest about their prior beliefs. So rather than using ad hominem attacks, or lying about how the data must have been collected, or implying that someone cheated, skeptics can show that the data aren’t strong enough to overcome their prior strong skepticism. The only problem is that the 2nd type of prior belief I mentioned above is not obvious to people who don’t understand statistics very well. So the skeptics can hide what they are doing that way, like Wagenmakers did in his analysis.
A final comment is that with enough data, the prior will have less weight and the data will have more weight. But it will take a very large amount of data to overcome the low prior belief of the debunkers.
I hope that’s helpful!
Jessica