The Quackometer

It is now one year since I issued a simple challenge to the homeopathic community to provide a simple and clear demonstration that what they say is true. If homeopaths could provide basic evidence that their beliefs are demonstrable, then much of the criticism of the trade would disappear.

But not one homeopath has been willing to give it a try. I bent over backwards to make it easy, cheap and accessible. Despite much bluster and hot air, on this site, and homeopathic discussion boards, no homeopath had the balls to come forward.


The test I proposed was simple: homeopaths claim that their pills have specific effects. In particular, each type of pill can induce specific and repeatable symptoms in healthy volunteers. (See the Society of Homeopaths explanation.) This is called a 'proving' in homeopathy and it is how homeopaths determine what each sort of pill is good for curing - 'like cures like.' For example, if a pill makes you feel tired when you are healthy it can be used to cure lethargy. Homeopaths routinely do provings on new substances, record the symptoms in groups of volunteers, and then add the new pills to their medical store cupboard. My test was simply this: given six different homeopathic pills, could a homeopath identify correctly which pill was which if all they know were what the six remedies were, but not which pill was which? The homeopath could choose whatever remedy they liked - to make them as distinct from each other as possible - and to take them in any 'strength' (remember homeopaths believe that the more dilute a substance, the more 'potent' it is). Just tell six pills apart. Simple.

I extended my challenge in a number of ways. I was quite happy for groups of homeopaths to do the 're-proving'. Some complained that this was too much of a burden for one homeopath. I was also quite happy for anyone to do anything to determine which pill was which. They could perform a re-proving, they could use any analytical technique (physical or chemical), they could dowse the pills or subject them to anything else they could think of. But not one homeopath wanted to end the controversy and prove homeopathy was real.


The reluctance of anyone to do this is fascinating in itself. But maybe I am making a big ask of homeopaths. Maybe it is a lot for just one or two homeopaths to do. And maybe it will take up more resources than I anticipate. Maybe it is more the sort of experiment that can only be realistically be done in a University. I don't believe any of this, but I want to give homeopaths the benefit of the doubt, and so I am now asking Universities to take up the challenge.

There are a small number of UK universities that offer a BSc degree in Homeopathy. These courses have been heavily criticised for being unscientific and not worthy of a science degree. At least one of the Universities is struggling with its course and is holding an internal review to see if it should continue to offer the course. I am asking these Universities to take up the challenge and encourage their students to do this test.

This ought to be easy. In any science degree, students spend many hours in laboratories, doing experiments, repeating the classic results that underpin their subject and learning about experimental technique and communicating their results. Why do BSc homeopathic courses not do simple tests like this as part of the learning programme? Why do students not take part in fundamental empirical tests of their subjects, like all other science degrees? If the universities want to deflect criticism of their courses, then surely demonstrating that their courses contain basic laboratory training in the fundamental scientific aspects of the subject would remove all criticism?


This test would undoubtedly be a great way for students to think about the scientific method and the nature of evidence. More than that, if the test was successful then it would be one of the biggest breakthroughs in homeopathy in 200 years. If such a test could be replicated across the Universities then I am sure criticism of homeopath would turn to amazement and excitement.


So, what I am proposing is the following:

1. I am writing to course heads and lecturers in five universities (details below) to invite them to take part in this test.


2. If they accept the test, I will post them six bottles of remedies. I propose to use standard High Street 30C remedies available from Boots, Holland and Barrett etc. However, the University may propose any remedies in any potency they like and I will source them from online homeopathic pharmacies.


3. The names of the remedies will be published on this site. The remedies will be dispatched in identical homeopathically ready bottles obtained from homeopathic supply companies. I will pay for the pills.
   
4. Each of the six bottles will only be identifable by a code letter (A-F).


5. I will post online an MD5 hash message digest of the code that relates each letter to each remedy with a salt to minimise attack. (Technical, but it means I cannot deny the results if they work out positive.)


6. When the university has completed whatever test it likes, I will post the code so that all can check this matches the hash and that I have not cheated.

I am quite willing to entertain all sensible proposals to modify this protocol as long as the statistical power of the test is not weakened and that the test remains blinded. I am happy for participants to try to make the test as easy as possible for them to pass in any way without compromising blinding.


And of course, the challenge is still open to any other individual or organisation that believes can prove homeopathy is not just plain sugar pills.


*************************************************************

In the morning, I am sending the offer to the following Universities and staff members:

Labels: $100 challenge, homeopathy

 

Homeopaths claim that their pills can induce predictable and distinct sets of symptoms in healthy people. That is how they prove their powerful medicine. This is basic stuff for homeopaths and my simple test asks them to prove it because I do not believe it for a minute. If this claim is not true then everything else they say falls apart. So, far no-one has the courage to come forward.

But I want to extend my offer. I have given homeopaths almost complete freedom in how they do this test. The only real basics I ask for is that the test is verifiably blinded and they do the test publicly, that is, say what they are doing in advance. I do not need to be involved.

Homeopaths are claiming that us sceptics are behind the times and there is now research from material scientists that show that homeopathic preparations are different from normal diluted solutions. I do no believe this too. I think the experiments so far can be explained by bad experimental design: either contamination or instrumentation problems. No chemist can reliably show how homeopathic water is different from ordinary water.

So, the extended challenge: exactly the same as before; any six remedies; some third party swaps the labels around; tell us which remedy is which when the labels are not on the bottle. You can use any experimental equipment you like, any conditions you like - you just have to be blinded. Simple.

As this is analytical chemistry, I think I would one further condition over the original challenge. The remedies must be 12C, or more potent. The claims of homeopaths are that post-Avogadro dilutions are recognisable.

Let's see it.

Any homeopathic friendly chemists want to do this? Your chance to punch this smart alec sceptic on the nose.

Labels: $100 challenge, homeopathy

 

Well, I have had two conversations with homeopaths now about taking the challenge. Recap: its a simple challenge to see if a homeopath can determine which remedy is which out of a sample of six when they do not already know which remedy is which. If the claims of homeopaths are correct, it ought to be easy.

First up: Sarah K, who left comments on my blog to say she was up the challenge. Fantastic news. Unfortunately, quite quickly she appeared to get rather defensive and say she would not be rushed into things. And ultimately decided that she did not want to do it. A huge shame.

Some of her excuses were that remedies could take a long time to take effect and might interfere with each other. Fine. But she has the choice to decide which remedies to use. For example, I suggested Nelsons Insomnia remedy. Customers of Nelsons might expect their insomnia to be reduced in a few hours, not over several weeks, and they would expect no lingering side-effects too, which homeopaths claim do not happen. Oh well. So close.

Another homeopath appeared to contradict Sarah K in the most striking terms. Soroush Ebrahimi was up for the challenge, but unfortunately not mine - he had his own ideas. He wanted to give me a remedy and be able to seal an envelope with a prediction of what the remedy would do to me - a 'reproving'. I think it best to allow the IM chat to speak for itself...

(for explanation, Soroush has already challenged Ben Goldacre to a similar test. This is IM, so spelling and sentence order may be erratic)


SE: Hi - did you want me for something? Welcome to my group of friends

LCN: Just wanted to know if you were prepared to take the quackometer $100 challenge. If you claim you could guess ben g's symptoms from taking a single remedy, then this test ought to be a walk in the park and great public proof of homeopathic claims.

SE: Hi - I did explain that the quackmaster test involves to elements:

SE: 1- Is the homoeopath any good?

SE: 2- Does a potentised substance work?

LCN: yes i am happy with that - do you think you are good homeopath?

SE: So if the result is a fail you do not know whetehr the homoeopath was poor in his selection or whether potentised substance have no effect

LCN: sure. but if you succeed then you win!

SE: So my 'reproving' exercise is better, because it elliminates the uncertainty about the ability of the homoeopath

LCN: no it does not. it relies on both the homeopath to interpret the symptoms correctly and the subject to report them correctly - both subjective and so unlikely to yield a good result for anyone. Eitgher side could cry foul. With my test, no one can cry foul.

SE: Neither with my test - when they are in agony and asking for help, then we will know for sure!

LCN: well frankly - if you are prepared to inflict agony on someone then you are not fit to conduct such tests. and ben was quite right to refuse.

SE: Oh - so dilute potentised substance do have an effect?? Make up your mind

LCN: so, you want to induce agony in someone and then claim that homeopathy is right? I think you do not understand anything about obtaining objective evidence

LCN: you are telling me they do. i think ben was quite right to worry tha tyou might poison him. i would be. my test involves no such risks.

SE: Ben was QUITE wrong - because if you read my post, you would have realised that he would have had the substance tested for himself and my half of the remedy would have been in a safe-box only to be opened with both of us present

SE: You guys have to put your body where your mouth is

SE: Either a potentised highly dilute material has powers or it does not

LCN: i am happy to take homeopathic remedes - but not from someone who promises to have me in agony! it shows a lack of ethics in the trial that you would be prepared to do so.

LCN: i have no doubt that the homeopathy would do nothing. I just would not be prepared to take a pill forom you.

SE: So tell me do potentised highly dilute substance have any power?

SE: if not - you are just taking a sugar pill as you lot claim

LCN: i believe they do not. but can i trust you to give me a potentised highly dilute substance ?

I hope you understand my reluctance to take part. The conversation went on for a while more. Mostly, me repeating a question to ask Soroush to take my test or work out a better protocol for his own.


For the record, I would take such a test, but Soroush needs to think up a protocol to take into account the following:

• S should not be allowed near any pill I would take.
• We need end measures that are not subjective - if the test fails I do not want homeopaths accusing me I was misreporting symptoms.
• We need a quantifiable result. At present, a success for S would not give us any idea of the significance of the result. Could it have been chance? We have no way of quantifying that.
• If it is going to cost more than my proposed test, I want to know why it is a better test and worth the money.

Without these things, Soroush's test is just a circus stunt. And, if I may say so, a very disconcerting circus stunts. Like a scary clown juggling sheep's hearts.

It is remarkable than no one else has come forward. This is a basic test of homeopathy. Something that ought to be easy and yet nothing quite like it appears never to have been done. It would show basic evidence that homeopathy is not a delusion. And yet, homeopaths feel no shame in taking on the responsibility for sick people. A responsibility that cannot be grounded in reason or evidence and instead relies on fragile anecdotes. Homeopaths' preference to counter criticism is to sue people rather than provide argument and evidence.

I am offering the chance for some homeopath to give me a metaphorical bloody nose, humiliate me, and prove me wrong - all for the cost of some postage and and few remedies.

So far, none of them have the courage of their convictions.

Labels: $100 challenge, Ben Goldacre

 

Homeopaths are feeling under threat at the moment and are scrambling around wondering what to do about it. I think there are a number of things they could do: most importantly, they should better manage their own business by showing that they respect the boundaries of what they can reasonably assume is good practice, e.g. stop the dangerous nonsense of believing that can do anything about dangerous conditions such as malaria and AIDS; they can be much more complementary and less alternative.

But there is something else that they can do too: start showing a desire to develop a base of data that can be relied upon, and respected, to support their methods. The focus to-date has been on clinical trials. Doubters say that trials show no evidence of efficacy. Supporters point to many positive trials. But it has been well documented that the many positive trials are most often poorly designed and reported, and are at best ambiguous in their results. There is not a compelling evidence base for homeopathy. If there was, there would be no argument.

So, let's take a step back. What sort of evidence would be required to convince me that there might be something in it? Fundamentally, my problem with homeopathy is its total implausibility - it contradicts what we know about the behaviour of matter. How can a plain sugar pill have any significant therapeutic effect on health? So, why not test the basic plausibility of homeopathy - can homeopathy pills do more than sugar pills in predictable way? There are a number of discussions about this going on in bloggerland and I would like to pick up on these and set a challenge...

Here is a rough outline of the sort of test I would like to see done...

1. A trained homeopath selects six homeopathic remedies of any type and strength.


2. The remedies are posted to an independent third party who removes the labels and replaces them with a code letter, A, B, C...F, and posts them back.


3. The homeopath takes each remedy in turn and notes the 'totality of symptoms'.


4. The homeopath writes down which remedy corresponds to which code letter.


5. The third party 'breaks the code' and we note how many are right.

1. Tell the world in advance that you are going to do this. Post your intention on a blog or web site, tell the world what you are going to do, be open to suggestions about how to simplify and make it a fair test. The more detail you publish, the more trust you will have. Remember, sceptics have a problem with trust of homeopaths.


2. State in advance what you think would be a successful result and any caveats you may have. Think of ways in which the trial may go wrong in advance, and make efforts to minimise those risks. None of us want excuses afterwards if it does not go well.


3. Find a genuine independent third party - someone with no stake in the outcome. Publish who they are and ensure they are happy to field questions from people after the trial. (People will want to know that protocol was followed).


4. Publish your results on the web before the code is broken to reveal how well you have done.

Labels: $100 challenge, homeopathy