In this season’s newsletter from the Homeopathy Research Institute, Dr Peter Fisher, Clinical Director of the NHS Hospital, the Royal London Hospital for Integrated Medicine, writes an essay on how homeopathy research may be being unfairly appraised due to the ‘plausibility bias’ of scientists.
Fisher makes a case that “Negative plausibility bias obstructs a fair
evaluation of the evidence around homeopathy”. That is, that views that homeopathy appears to be ‘impossible’ clouds the judgements of people when reviewing scientific studies into the effects of homeopathy. Indeed, Fisher claims that some trial data for homeopathy has similar levels of evidence to mainstream treatments, but that the lack of a “plausible
theoretical framework” means that the these trials have little impact on medical practice.
The article is written for homeopaths as a summary of a paper that Fisher co-authored where they discuss the impact of their concept of ‘plausibility bias’ on how the trials of homeopathic treatment of upper respiratory tract infection. The authors make a case that “the same level of supporting clinical trial evidence should be accepted for all scientific developments”.
Fisher quotes Chaplin, a researcher into the Memory of Water,
If a lower level of proof is set for hypotheses that fit prior beliefs then we bias our view of science in favour of such beliefs and may be easily misled.
To back up this claim, Fisher adds,
‘Inherent implausibility’ is a poor guide to future understanding. History is littered with examples of ideas that at one time appeared highly implausible but are now accepted as fundamental truths: the Copernican revolution and quantum physics are well –known examples.
Now, before I show why that last statement is a confusion, it is worth looking at this charge that sceptics of homeopathy, like myself, have a negative bias against the clinical results of homeopathy because we have no ‘plausible framework’ for how it might work. Indeed, this is a common charge laid against critics that because ‘we don’t know how it works, does not mean it does not work’.
There are several confusions here and many a big conflation. But the argument that we should treat all treatments the same has a superficial appeal. Surely, as dispassionate scientists we should not ‘load the dice’ against treatments that we may find problematic for some reason?
It is of course nonsense. Even if you held this view about homeopathy, it is not how you view your everyday beliefs. We all work on the natural assumption that if a claim is unlikely then we will need extra evidence to change our minds.
Let’s imagine a hypothetical homeopath sat at home in front of the telly.
In rushes their five year old son saying that they have seen a dog in the garden. The homeopath’s belief prior to this event is that there are no dogs in the garden. They do not own one. However, should the homeopath now change their mind when this new piece of evidence comes along? Their five-year-old does make up stories sometimes, but it could be true. Let’s say 50-50 for now. So it is worth investigating further and so, the homeopath gets up and wanders out into the garden where they see a bush rattling as if a large creature is in there. Should the homeopath upgrade their beliefs about the number of dogs in the garden? It would be wise to do so. It is not clear what else could be disturbing the bush so much. So the homeopath investigates further and finds next-door’s Fido, who has escaped and is burying a bone in the bush. The homeopathy is now 100% certain that there is a dog in the garden and can act appropriately.
The plausibility of the claims and the nature of the evidence allows the homeopath to rapidly upgrade their confidence in their child’s hypothesis that there was a dog in the garden.
Now consider a second example. This time the child rushes into the house claiming there is a tiger in the garden. This being Southern England, the plausibility of there being a tiger in the garden is near zero. Not impossible, but near zero. So, the homeopath will not hedge their bets and call the police to send a marksman, but will probably wander out with their child asking what stories they are making up now? But a bush is seen rusting too. This is the same level of evidence we had for the dog, but the homeopath is not going to treat the claims identically because the prior probability of it being true is quite different. As it turns out, the escaped tiger turned out to be a toy. And coincidently, Fido was after his buried bone in the bush. So, no tiger. It was a preposterous children’s invention.
We naturally appraise claims and evidence on their plausibility. The new facts would have to fit in with what we already know about the world. The world is not capricious. If a claim flies in the face of what we know, we do not change our mind until the evidence is overwhelming.
So, counter to what Martin Chaplin says that we ‘may be easily misled’ if we ask for greater evidence for ideas that do not fit in with our prior beliefs, the exact opposite is true. Indeed, we call people who too readily accept wild claims gullible, credulous and fools.
Science codifies this thinking into a statistical formula. Indeed, Peter Fisher, in his paper discusses the role of Bayes’ theorem on how we should change our mind in the face of new evidence. This statistical technique tells us that if we have a certain confidence in a belief, then when a new piece of evidence comes along, how much should we change our mind?
Fisher’s argument in his academic paper is a little different from the homeopaths’ report. In the paper, the authors fully acknowledge the role Bayesean analysis plays in affirming hypotheses. Their argument is simple though: that in order to apply Bayes properly, you need to understand the prior probability that your hypothesis might be true. In other words, in the absence of trial data, what confidence can you have that the claims of homeopathy are correct?
This is indeed a weakness of using Bayes theorem in real life. What probability would you put on there being a tiger in the garden? Tigers have escaped from zoos before, but it is quite hard to come up with a number. Some might think that for homeopathy it is more like asking the probability that there is a dinosaur in the garden. Is such a thing truly impossible? Or could some Jurassic Park like experiment make this a possibility one day?
However, for homeopathy, the situation is probably more like asking if there is a fairy or dragon in the garden. As Fisher quotes, “Accepting that infinite dilutions work would subvert more than conventional medicine; it wrecks a whole edifice of chemistry and physics”. Homeopathy is not really just implausible, it is impossible according to very well established principles of basic science.
Applying Bayes in this context, then indeed does become deeply problematic. But the problems run deeper than even Fisher appears to think. Whilst we might argue that the prior probability is zero on the basis of the impossibility of the claims, Fisher argues that the probability is finite – and indeed, larger than mainstream scientists would accept – and this is the source of the plausibility bias. We shall come onto this.
But first, back to the claim that science does occasionally support implausible claims. This would make the chance that homeopathy is true to be finite if indeed it is true that “History is littered with examples of ideas that at one time appeared highly implausible but are now accepted as fundamental truths”.
However, I would argue that the examples Fisher gives do not support this very well. (Indeed, an aside challenge is for anyone to come up with a good example where science eventually confirmed a genuinely implausible hypothesis. )
Fisher says that “quantum physics” is a good example. I do not agree. Quantum theory arose because experiments were turning up very strange results. In 1905 Einstein published a paper on the photoelectric effect that showed that this phenomenon could be explained if light behaved as if it consisted of discrete particles carrying specific energies. This was not implausible, but highly counterintuitive. The prevailing concepts of light involved it behaving as a wave. For it to behave as ‘quantised’ particles ran counter to what was seen in other experiments. From Einstein’s early work came the full theory of Quantum Mechanics. Despite it being a highly successful theory, it still remained highly counterintuitive and indeed Einstein did not like what it appeared to be saying.
Science is often counterintuitive in that its results often appear to contradict what we think we know from everyday experience. Implausibility, however, is where a result appears that we thought was highly improbable due to well characterised principles.
The example of the Copernican Revolution also does not convince me that implausible theories are often found to be true. In 1543 it too might have appeared to be counterintuitive that the planets revolved around the Sun. But was it also implausible (or indeed impossible) in the same way the hypotheses of homeopathy are implausible? Homeopathy runs counter to current, well established science. The Copernican Revolution ran counter to Ptolemaic models of the Universe. But these were not well established scientific theories as we know them to day, but ideas that were based mainly on Greek authority and the insistence of the Catholic Church that the Earth was at the centre of the Universe. The Copernican Revolution was more of a move away from dogma than the triumph of a theory over its own inherent implausibility.
But even if you were to argue that there have been implausible ideas that turned out to be true, it naturally does not mean that implausible ideas will be shown to be true. An implausible idea is exactly that – and the number of implausible ideas that can quite safely be forgotten will always greatly outnumber those that turn out to be correct.
So, by appealing to the idea that we cannot dismiss homeopathy because other ideas that appeared to be crackpot eventually came good, Fisher is obviously on very dodgy ground. So, his main thrust is to push the idea that the implausibility of homeopathy is not well founded at all.
Fisher resorts to portraying the critics of homeopathy as refusing to accept the number of replications of basic experiments into homeopathy and the ideas of homeopaths as to how this implausibility might be overcome. We are back on familiar territory now as Fisher makes a number of well exercised misrepresentations.
The House of Commons Select Committee review of the evidence for Homeopathy comes under fire first. This is important as the Advertising Standards Authority use it to justify asking homeopaths to remove all health claims from their advertising and web sites.
This report was heavily criticised, particularly for its failure to take evidence from a single patient who had experienced homeopathic treatment and from only one practitioner (me), while calling a number of well-known sceptics including representatives of Sense about Science, a lobby group which has campaigned stridently against homeopathy
It is not true that only one practitioner gave evidence or that patients were ignored. The report is an inch thick and full of the written evidence from many homeopaths, users and scientists. Oral evidence was only given by a select few, including homeopathic manufacturers – but all evidence was discussed in the report.
He continues to suggest that basic science is now replicating results that confirm a homeopathic action,
The best established method utilises the Human Basophil Degranulation Test – a test tube model of allergic response. The finding that homeopathic dilutions of histamine inhibit basophil degranulation has been verified repeatedly by different scientific teams.
A paper by Endler, publish in 2010 is used to support this claim, Repetitions of fundamental research models for homeopathically prepared dilutions beyond 10-23: a bibliometric study.
Edler says of the Basophil experiments, “However, when comparing the studies in detail one must
conclude that no independent repetition trial yielded exactly the same results as the initial study, and methods always differed to a smaller or larger extent.”
This is hardly a statement that is going to lead to a conclusion that the Basophil experiments are capable of consitent replication. What is even more surprising is that Peter Fisher did not use another review that concentrated exclusively on the basophil experiment to back up is statement.
Madeleine Ennis published her review also in 2010 and concluded that,
The methods are poorly standardized between laboratories – although the same is true for conventional studies as described above. Certainly there appears to be some evidence for an effect – albeit small in some cases – with the high dilutions in several different laboratories using the ﬂow cytometric methodologies. How much of the effect is due to artifacts remains to be investigated.
Again, inconclusive and in need of definitive work. This paper was published in Peter Fisher’s own journal Homeopathy. Why Fisher did not quote this is not clear. I note the Edler paper is more positive in its editorial.
Finally, Fisher tries to tell us that
Beyond this is the question of how these effects are mediated. Although the work is preliminary, many believe that ‘nanostructures’ in water may be involved. Supporters of this view include the Nobel Laureate, Luc Montagnier, who has published remarkable results supporting this hypothesis, although these await independent replication.
This is just gobbledegook. There is no coherent explanation that involves ‘nanostructures’ that can rescue homeopathy from the very deep well of impossibility. There are a number of fringe scientists and supporters of homeopathy who propose different entities that somehow survive the extreme dilutions to be the carriers of the homeopathic effect. But no one has describes how these structures themselves overcome the inevitable dilutions that prevent the original substances from ending up in the pill. Rather than solve any homeopathic mystery, so-called ‘nanostructures’ merely pile on new layers of implausibility.
We are treated to an appeal to the authority of the Nobel Laureate Luc Montagnier as a last gasp to present some plausibility. Montagnier, whose research does not mention homeopathy, has so far failed to publish any of his high dilution work in independent peer reviewed journals. Is it plausible that Montagnier is correct with his outlandish claims, or is it more reasonable to see a scientist who did his best work a quarter of a century ago and is now pursuing a folly as he approaches retirement?
Evidence based medicine is built on an important assumption: that the prior probabilities of treatments under test can be considered to be constant. That is why most of the time, no account is taken of the underlying mechanism, at and that the clinical trials can speak for themselves. It is assumed that any treatment being tested on humans has a reasonable chance of success and is not based on non-physical or anti-scientific thinking. But this assumption breaks down for superstitious and pseudoscientific forms of treatments where the prior probability may be exceptionally low. Fisher is asking us to ignore this problem and just run with evidence based medicine as if there were no issues with the absurdity of treatments.
But extraordinary claims require extraordinary evidence. And rather than it being mainstream scientists that are suffering from a plausibility bias when considering homeopathy, the reverse is true: homeopaths, ignore the science that suggests their treatments are impossible and ignore the highly plausible reasons as to why they may be tricked into thinking an inert treatment is effective.
Fisher is right to conclude that studies of homeopathy “may have limited impact on practice until a plausible theoretical framework is established”. What he cannot accept – because of his own plausibility bias – is the extreme unlikelihood of that ever happening.
Fisher claims that some trial data for homeopathy has similar levels of evidence to mainstream treatments, but that the lack of a “plausible theoretical framework” means that the these trials have little impact on medical practice.
Maybe he’s forgotten Sagan’s words of advice, “Extraordinary claims require extraordinary evidence.”
“And rather than it being mainstream scientists that are suffering from a plausibility bias when considering homeopathy, the reverse is true”
Exactly right. Moronic, deluded homeopaths claiming that black is white again. &_&
“Applying Bayes in this context, then indeed does become deeply problematic.”
Not really. If we can find about 100dB of evidence against homeopathy we’re done¹. There’s certainly no excuse for doing futile and unethical CTs of it. And if we can’t find at least that much we should probably divert most of the LHC budget to phlogiston research etc. too.
¹ “An hypothesis A that starts out down at 100 db can hardly ever come to be believed whatever the data, because there are almost sure to be alternative hypotheses (B1 ; B2 ; : : 🙂 above it, perhaps down at 60 db. Then when we get astonishing data that might have resurrected A, the alternatives will be resurrected instead.” –Jaynes.
Fisher is right. I have had terrible trouble getting NASA to properly investigate the moon-cheese theory; they claim that the idea the moon is made of green cheese is “implausible” so they flatly refuse to undertake the research necessary to prove the case.
Does any one know if the consent form signed by patients undergoing treatment at the Royal London Homeopathic Hospital (aka RLHIM)informs them the treatment is implausible by modern scientific standards? Dr Fisher (regulated by the GMC) must do his best for his patients, not simply foist them with his unsubstantiated opinions. He is a physician, not a priest.
Many patients might not be too concerned, but some might be. All patients should have a choice in the matter – informed by the best available evidence presented in a way that is comprehensible to them.
Scientists do not need to research into ‘homeopathy’ – for that is a belief system and a matter for psychologists and philosophers, if not theologians.
They might research into ‘The effects of dilutions of chemicals beyond Avogadro’s number on pathological or physiological processes in animals’. Or some such.
Such ‘research’ on animals would be unethical.
Homo sapiens is an animal.
And what is un-ethical about giving an animal water containing no chemically or physically active ingredient?
The control group would of course be given water containing, well, water.
Nothing – if the animal is thirsty. On the other hand there are ethical problems with giving an animal water as a treatment if there is an effective treatment for the animal’s condition. It isn’t as if the animal will benefit from the placebo effect – any apparent “improvement” will be reported by the owner or vet, not by the animal. If trials of homoeopathy in animals are to be considered, the issue of plausibility arises again.
Judging by the content of the hospital’s various information leaflets promoting everything from homeopathy to CST, I’d say the answer to your question is likely to be ‘no’. (I’ve submitted complaints about some of these to the ASA.)
There are some additional problems with Fisher’s arguments, leaving aside the most obvious one that the empirical evidence for homeopathy is, at best, inconclusive. Of course, labelling it as inconclusive is very, very generous.
– “Fisher claims that some trial data for homeopathy has similar levels of evidence to mainstream treatments.”
Perhaps so; but this means only that the evidence for some “mainstream” treatments is poor. A call for more evidence-based medicine, not for less!
– One of the problems with Fisher’s examples is that he is quoting them with hindsight. That’s selection bias at work: he forgets the many implausible ideas that later turn out to be wrong.
– The next problem is that his examples weren’t implausible at all. In fact, the ancient greeks were aware of the idea that the earths was a sphere and rotates around the sun, they even came up with a pretty good estimate of the earth’s size. The hypothesis didn’t establish itself agains competing models, but that’s another story… The explanation might be that the ancient greeks cared more for philosophical debate than for empirical evidence.
The second example is not very convincing, either. “Light as particles” wasn’t a new idea in Einstein’s days, it was championed by none other than Newton but later fell out of favor.
But in line with the categorisation of tials used by the BHA/FoH (and frequently cited by other homoeopaths) in which trials that show no significant difference between placebo (or other comparator) and homoeopathy are categorised as “statistically non-conclusive”, with only those in which homoeopathy performed significantly worse than comparator being categorised as “negative” (see Memorandum submitted by The British Homeopathic Association (HO 12), for example).
Generous to a fault. In reality there is zero empirical evidence /for/ homeopathy and a conclusive mountain of evidence /against/ it. CTs of it are unethical¹ cargo cult science and aren’t evidence of anything except delusion, ignorance and stupidity.
Uh… I forgot the link to the ethics article:
(written by one of the apparently few people who seem able to transcend their “scandalous” orthodox stats mis-education and understand the inferential logic which renders CTs of homeopathy futile non-science.) /snipe 😉
At least Fisher hasn’t yet claimed that homeopathy is no more implausible than Ohm’s Law once was…
“Ohm’s law was probably the most important of the early quantitative descriptions of the physics of electricity. We consider it almost obvious today. When Ohm first published his work, this was not the case; critics reacted to his treatment of the subject with hostility. They called his work a “web of naked fancies” and the German Minister of Education proclaimed that “a professor who preached such heresies was unworthy to teach science.” The prevailing scientific philosophy in Germany at the time asserted that experiments need not be performed to develop an understanding of nature because nature is so well ordered, and that scientific truths may be deduced through reasoning alone. Also, Ohm’s brother Martin, a mathematician, was battling the German educational system. These factors hindered the acceptance of Ohm’s work, and his work did not become widely accepted until the 1840s. Fortunately, Ohm received recognition for his contributions to science well before he died.” http://en.wikipedia.org/wiki/Ohm%27s_law#History
Of the top of my head (and therefore possibly mildly garbled) an example of an initially implausible theory is evolution:
Darwin’s theory of evolution requires huge amount of time. When he published the theory the limit on the age of the earth was imposed by the age of the sun (as it still is). Fusion was unknown at the time, and the best guess for the source of the sun’s energy was its gravitational collapse. Calculations by Lord Kelvin et al assuming this source of energy made the solar system too young to provide nearly enough time for evolution. Rendering evolution theory “implausible” until something new, fusion, came to the rescue.
I accept that evolution is so wonderfully explicative that you might take it as a reason to reconsider the age of the sun rather than use the apparent age of the sun to dismiss the theory of evolution.
Of course examples such as this help homeopaths not at all. They all show that science has happily advanced by overturning previous ideas many times since Hahnemann. The reason science hasn’t adopted homeopathy isn’t because science is closed minded, but because homeopathy is a load of shit.
Kelvin’s result whilst interesting wasn’t accepted by many Geologists who thought the world was older.
I’m guessing the geologists had evidence for the earth’s age, presumably looking at things like the Grand Canyon where you see 17 million years of erosion through 2 billion years of rock, which has been rising for 65 million years, gives you some hints that 400 million upper limit was wrong.
Be interested to know if anyone is aware how much faith was placed in the different arguments at the time, as plenty of people came up with divergent estimates of the earth’s age. I know Darwin was aware of Lyell and others who realised the rocks were really OLD, including the ones with fossil snails in.
I wonder if the idea making the rounds currently that old practices disappear when the current generation retires will apply to homeopathy. Hopefully this is true. It will take continued pressure for years to achieve this.
I remember a comment from a young, “former” practitioner of homeopathy who had fled the field. Perhaps he is the start of a retreat. Hope so.
Hahnemann has been dead for a while.
Invoking Bayes’ Theorem wouldn’t seem to help homeopaths. Even if you set the prior plausibility of homeopathy being effective at 50% or even 100%, once you started looking at the data, repeated instances of the null result would soon reduce the plausibility of homeopathy down to near zero.
Some implausible ideas that have come to be accepted:
(a) Wegener’s “Continental Drift” hypothesis was viewed as extremely implausible by a large number of geologists for a considerable time, as they could not see a mechanism by which continents could plough across the ocean floors. (As it turns out, the ocean floors spread and carry the continents with them.)
(b) During the middle third of the 20th century, the Big Bang theory was viewed by many cosmologists as being implausible when compared to the Steady State theory. The steady state picture of continuous creation seemed much more likely than the Universe being “poofed” into existence out of nothing.
(c) Parity violation in the weak interaction – until this was discovered, physicists assumed that the laws of physics must be left-right symmetric. I understand that everyone was astonished when this turned out not to be the case.
In fact, I think that science has no trouble with accepting ideas initially felt to be implausible so long as enough experimental evidence is forthcoming. Peter Fisher and his friends should simply keep on doing the experiments, publishing the results, and trying to explain what they find out. But they must surely realise that calling homeopathy “implausible” really is a massive understatement. Even if we accept that stable “nanostructures” exist in water (and I’ve seen no good evidence for this whatever), this is far from establishing any part of homeopathy. He would still need to answer:
(i) Why should continued dilution and succussion make nanostructures stronger?
(ii) Why should nanostructures have a biological effect, and why should this effect be related to the original substance that has been diluted away?
(iii) What are the effects of time and temperature upon the stability of nanostructures?
(iv) What nanostructures are present in ordinary drinking water?
(v) If water containing nanostructures is dripped onto a sugar pill and then allowed to evaporate, are the nanostructures re-created when the sugar pill is put into fresh water?
It may be that every such question has a satisfactory answer and that the majority opinion is completely wrong. However, at present it is hard to take seriously anyone who thinks that homeopathy has achieved scientific respectability. Even the most ardent supporter of homeopathy must admit, surely, that they have a lot of work still to do.
“Even the most ardent supporter of homeopathy must admit, surely, that they have a lot of work still to do.”
In the prison laundries perhaps. Homeopathy is absurd and blatantly false nonsense and there is exactly no “work still to do”. Examples of later confirmed ‘implausibile’ ideas – like yours and Matt’s – are all very well but even when they’re not clearly subjectively tainted (like the extreme Ohm’s Law example), they fail to distinguish between the *mild* implausibility caused by contradiction of conservative extrapolation to unprobed regions of things already known to be true in some domain, and the *extreme* implausibility caused by claims that are in contradiction of things which are firmly established knowledge in some region of some domain. Kelvin must’ve (or should’ve) known that he had no idea what was and wasn’t physically possible in the regions of pressure and temperature in the sun.
All this brings to mind the convoluted, and very illogical, reasoning medieval theologians employed to prove the existence of god by argument alone. They started out by believing god existed. All the arguments they used, therefore, had to be consistent with this ‘fact’.
Homeopaths are doing the same thing. They start off by believing homeopathy works, and everything they say is based on this belief.
It would all be very entertaining and amusing if it weren’t so dangerous. Any sane and rational person would think a modern, scientific society would automatically ban substances that haven’t been scientifically proven to have any efficacy. But, then, some people still believe in god.
Please don’t seek to ban homeopathic remedies.
IMHO folks must be able to be foolish if they so choose.
But the choice must be an informed one, and they need to know Homeopathy is generally regarded as regarded as ‘implausible’, ‘alternative to reason’ and every other negative epithet you can think of.
We must try to educate them, but if they ignore us (and reason), so be it, PROVIDING they do no harm to other people.
And so they must NOT expect non-believers to pay for their beliefs. No homeopathy on the NHS, nor funded by charities which support rational medicine. That would harm other people by using resources which otherwise would be available for rational medicine.
The believers must not act in such a way as to take advantage of the gullible. So they must expect, and welcome, their marketing being subject to regulation by ASA and others.
For the same reason they should NOT expect or countenance attempts to ‘integrate’ their beliefs with rational medicine. That would harm the rationality of medicine by diluting it.(And they know all about dilution and how powerful it is).
Yes, I know orthodox medicine is not entirely rational, but it does at least try to be, and we all do our best. And we move on when evidence suggests we should.
You mean, declare homeopathy a religion and be done with it? 🙂
On the issue of implausibility, theoretical physics is surely just one of several problems. There’s also the question of how a non-existent quantity of arsenic, in a sugar pill or water, can supposedly kick-start the body’s natural healing response to arsenic poisoning when the arsenic poisoning itself did not invoke a similar response.
It’s like healing a gunshot wound by shooting the victim with blanks. It might work but seems implausible. Maybe we could arrange some tests.
All the above is correct — and a nice exposition of the intuitive take on Bayes’ Theorem! @Michael5MacKay makes a good point in that Bayes priors only give you a starting point — any unfavourable prior can be overturned with sufficiently strong data, a.k.a. the Sagan “extraordinary evidence” maxim.
But I suspect this is all a bit irrelevant, even when taking into account that CAM practitioners have not in the past shown themselves to be amenable to rational arguments and that this latest attempt is just another attempt to muddy the argument with some technical sounding nonsense about statistical priors and quantum mechanics…
If CAM treatments, and homeopathy in particular, had been shown in any reasonable collection of well-controlled trials to have a definitive effect beyond placebo, then Peter Fisher would have a point. “Why does homeopathy have to display a stronger definitive effect than conventional medicine^W^W allopathy to be accepted?” he would say, and there would be something to that. Even without the micro-mechanism being known, it would be a very interesting result. But as far as I’m aware, the situation is nothing like that — conventional interventions with trial stats as poor as CAM should not (and would not? I may be wrong, in which case I’d love to see the examples) make it to market. The evidence has not even got close to the stage where concern over priors enters: dumb old “anterior” statistics already says it’s a crock.
Wish I could “like” comments on this page, Andy (and indeed refer the articles themselves to various social networks) — some excellent observations.
“any unfavourable prior can be overturned with sufficiently strong data, a.k.a. the Sagan “extraordinary evidence” maxim.”
This Bayesian stuff is much more interesting than homeopathy. However, phayes, I do think you’re fundamentally wrong on this point. The prior probability in this treatment is nothing more than a numerical expression of your confidence in being correct. The possibility that one is entirely confident and entirely wrong always remains. Suppose there is an as yet unimagined theoretical framework in which all of our current understanding of matter is but a special case of a more general theory which includes the possibility of homeopathy (joke!)? You can’t know there isn’t just because you understand our current theory.
The point is that Bayesian analysis works with everything you currently understand, plus the next bit of information. The purpose of science is to test hypothesis to destruction and in so doing find the piece of information you don’t know that you don’t know.
The Bayesian approach is also a bad strategy for convincing others for two reasons.
Firstly it looks to the uninitiated like an esoteric justification for dogmatic certainty, this is unattractive.
Secondly it lets the homeopath get away with a year zero fallacy, which is exactly what Fisher has done again in the article described above. They give the impression that they’ve suddenly happened upon the world with a wonderful new medicine only to be spurned by the closed minded scientists they found there. The reality is they are the last remnant of humeral medicine based on the wholly false idea that the body is composed of metaphysical fluids, an imbalance in which causes all disease. It is they who have ignored the total revolution in understanding of biology and medicine in the last 200 years. We should make that case more strongly.
“However, phayes, I do think you’re fundamentally wrong on this point.”
I’m not 🙂 and you’re missing the key point and assailing some unassailable logic. The prior probability I assign to homeopathy is not “nothing more than a numerical expression of [my] confidence…”, it is a measure of the evidence against homeopathy. The fact that I don’t know *for sure* that homeopathy is impossible is irrelevant: the key point I’ve been trying to make – and Jaynes makes – is that all that matters is that I do know for sure, in the context of an experiment such as a CT, how the homeopathy hypothesis compares plausibility-wise against the spectrum of alternatives which are usually and justifiably neglected in applications of inference to empirical science. The inescapable consequence is that in the event that some homeopathy CT produced (even extraordinary) positive results the rational interpretation and causal explanation of those results would be error, not homeopathy.
“The purpose of science is to test hypothesis to destruction and in so doing find the piece of information you don’t know that you don’t know.”
That’s not true. The purpose of science is to gain reliable knowledge about the world in an efficient manner. That doesn’t include testing unmotivated established science-defying ‘hypotheses’. The formulation and selection of hypotheses is at least as important a part of science as the testing to destruction of them.
I will now assail your unassailable logic 🙂
Consider the example from your pdf of Laplace and the meteorite, which I will unashamedly muck about with and render historically inaccurate to make a point.
So there is Laplace, sat in Paris, confident that rocks are very much ground based things and so putting an infinitesimal prior on rocks falling out of the sky. Then a rock is reported to fall out of the sky at Laigly by reputable sources, an example of extraordinary proof, and he changes his mind. If I understand your logic, you think him wrong to do so. He must be so confident in his prior belief that no posterior could move it. He should also be able to formulate many hypothesis, examples below, more plausible than rocks in the sky, and so remain unmoved by the events at Laigly.
H1 Sources are lying
H2 Group hallucination
H3 It’s a prank with a firework and a catapult
H4 It’s actually an example of volcanic activity
Still, suppose a meteorite hadn’t landed at Laigly, what then. Suppose Laplace finally gets round to reading Newton and discovers a compelling theory that unites the ideas of rocks falling to the ground on earth, but travelling in curved paths through the heavens. Suddenly, without any new data, Laplace alters his assessment of his prior from near 0 to near unity. Now he is likely to accept reports of meteorites with equanimity.
In the first case Laplace could have constructed a false, but for him unchallengeable intellectual position, using your logic (correct me on this if I’m wrong). The second case is an example of how Laplace’s first prior was based on a false assessment of the validity of his early theory of rocks, corrected by a better theory. In your words his “measure of the evidence” was wrong.
Finding a good “measure of the evidence” is of course what science is about, but then you have a debate about how to assess the available evidence to set the prior. In which case the Bayesian formalism at best tells you what you already know, and at worst leads you to beg the question.
I agree that the selection of hypothesis is important in science and that homeopathy is not an interesting subject for biological science. However its widespread application puts people at risk, protecting people from it requires something more persuasive to the general public than your Bayesian treatment.
“So there is Laplace, sat in Paris, confident that rocks are very much ground based things and so putting an infinitesimal prior on rocks falling out of the sky.” etc.
I give up.
I demonstrate the application of Bayes’ theorem by putting 10 red poker chips into a cloth bag, then 10 yellow. A shake and an audience member draws one out.
We record what it is. 50:50 it is red for the first draw.
But then as we draw more and more the next eight are also red!
So now what are the odds on the next draw picking the last red or one of the original 10 yellow?
Only then do I point out that I am a member of the Magic circle, so the likelihood is 1.0 (100%) that the draw will be whatever I want it to be!
Had they had prior knowledge of my magical status, they probably would not have suggested the liklihoods they did.
Well, it livens up an otherwise boring lecture on stastics!
If you ain’t got all the information, be very wary of an outcome.
“The best established method utilises the Human Basophil Degranulation Test – a test tube model of allergic response. The finding that homeopathic dilutions of histamine inhibit basophil degranulation has been verified repeatedly by different scientific teams.
A paper by Endler, publish in 2010 is used to support this claim, Repetitions of fundamental research models for homeopathically prepared dilutions beyond 10-23: a bibliometric study.”
Even staring with a 1 molar solution of histamine a dilution of 10-24 and beyond will provide a solution without one molecule of histamine and at a dilution of 10-26 and beyond not one molecule of water from the original solution. This is not complex science just simple maths.