The Peninsula College of Medicine and Dentistry at Exeter and Plymouth Universities have issued an excited press release about how their researchers have demonstrated the benefits of acupuncture and how it would save resources for the NHS.
They claim that ‘5 element acupuncture’ has been shown to be effective with people suffering from ‘unexplained symptoms’. A paper was published in the British Journal of General Practice that concluded that “acupuncture had a significant and sustained benefit” on such people.
Except the paper does not show any such benefit.
Even a cursory glance at the data in the paper shows quite clearly that patients received no significant benefit from acupuncture against a control group (who received delayed acupuncture). This is somewhat surprising as the study was not properly controlled – patients and practitioners were not blinded – and you would have expected to see significant biases creep in, suggesting an illusory effectiveness. This might suggest that acupuncture treatment does not even provide a placebo benefit.
DCScience takes the paper apart here. Margaret McCartney looks a the absurd conclusions too.
Despite the paper clearly showing no effect, the authors bizarrely conclude that the acupuncture had been effective. Patients did get better, but no meaningful difference was seen between groups. But surely to the delight of the researchers, the positive, misleading conclusions created headlines in the newspapers. (Uncritical pieces by Jenny Hope in the Daily Mail and the Telegraph.)
We should assume that the authors have not been deliberately dishonest in their misrepresentation of their own work. Advocates of acupuncture and other alternative medicines have powerful beliefs in their own practices. But why does this happen?
Just as there are optical illusions that trick us into seeing things that are not really there, so there are health and medical illusions that trick us into believing treatments are effective when they are not. Expectations of effects can create false beliefs and sensations in suggestible people. Because symptoms might improve after a treatment, an illusion can be created that the improvement was caused by the treatment when such an improvement might have happened anyway. The entire world of alternative medicine can be explained by such illusions – and they are very powerful. Our imaginations play powerful tricks around our experiences of illness.
Whereas we may not be very embarrassed to be tricked by an optical illusion, we appear to be very resistant to the idea that we have been tricked by a health illusion. Perhaps this is because of the level of investment we put into such beliefs. We create hope from them, define ourselves by our acceptance of them, and sink money and time into their pursuit.
Practitioners who have invested their lifetimes in illusory practices may well find it very hard to accept contrary evidence, no matter how strong that evidence is.
One of the first persons to realise the role of illusion in medicine was a doctor from Bath by the name of John Haygarth. Bath in the late eighteenth century was the centre of the ‘Golden Age of Quackery’. Whereas the wealth of nearby city Bristol, as displayed by its extensive and elegant Georgian terraces, was acquired through its seafaring trade (significantly in slaves), Bath’s even more majestic houses were built on the wealth generated from trade in the illusory curative powers of its hot springs. But the ancient sacred waters of Roman Bath were not the only curative show in town. The huge numbers of health tourists attracted all sorts of charlatans and quacks to take money from eager patients.
One such miracle cure were the ‘tractors’ of American father and son, Joseph and Elisha Perkins. The tractors were 3 inch pointed metal rods that were supposed to pull off “noxious electrical fluid that lay at the root of suffering”.
The tractors were made of a secret alloy and were pressed and rubbed into the effected part of the body. In describing their method of action as the manipulation of electrical fluids, Perkins was tapping into the current excitement surrounding Galvani’s recent discovery of “animal electricity” and the popularity of Mesmer’s “animal magnetism”.
Coming to England and charging 5 guineas for a pair of tractors, a substantial sum at the time, the Perkins were quick to make a fortune on the back of a tractor craze. In London, Lord Rivers set up the Perkinsian Institution “for the benefit of the poor”.
According to Haygarth, in Bath, the tractors gained a ’high reputation’ as a cure for all sorts of ailments, with patients reporting miraculous improvements and during tractor sessions would feel tingling and heat as the tractors were applied.
But Haygarth was obviously worried by the popularity of the devices and also sceptical of the bizarre nature of their application. He wrote,
In physick [medicine] as well as philosophy, to discover the cause of an effect, is frequently a very nice and difficult investigation. The success of medical practice depends on this faculty of sagacious discernment.
Clearly, Haygarth recognised that simply hearing a patient report an improvement does not mean that the particular therapy being used was the cause of the improvement. To show that this is so is not a trivial problem. Doctors who jumped to conclusions such as this on the back of whim risked harming their patients,
One who is carried away by every wind of doctrine, and who recommends every new medicine which is in vogue, and becomes popular without considering the evidence and fair probability of efficacy, will frequently administer inert and sometimes injurious remedies to his patients.
Haygarth set about conducting an experiment to see if the metallic tractors were the cause of the observed effects.
Let their merit be impartially investigated, in order to support their fame, it it be well founded, or to correct the public opinion, if merely formed upon delusion.
To do so, he constructed his own tractors, but made of wood.
Prepare a pair of false, exactly to resemble the True Tractors. Let the secret remain inviolable, not only from the patient but from every other person. Let the efficacy of both be impartially tried.
What he observed was that is made no difference as to whether the metallic or wooden tractors were used. Indeed, he showed that you could use just about anything, including bones, pencils and tobacco pipes. Patients appeared to be imagining that the therapies were having some effect.
What Haygarth had discovered was far more important than tractors were ineffective treatments. He recognised that it was important to separate out the specific effects of any treatment from effects caused by “the passions of hope and faith, excited by mere imagination”.
This method of discovering the truth, distinctly proves to what a surprising degree mere fancy deceived the patient himself.
He warns us in a way that would benefit researchers into acupuncture that,
If the experiment had been tried with metallick tractors only, they might and most probably have deceived even medical observers.
Haygarth was proposing the foundations of the modern randomised controlled trial and had established the utility of key elements, namely comparison against an inert and indistinguishable control and the blinding of subjects to which treatment they were receiving. Neither of these important principles were employed by the acupuncture researchers.
Haygarth was able describe the mechanism by which quackery can appear to be effective.
On this principle we can account for the marvellous recoveries frequently ascribed to empirical remedies which are commonly inert drugs, and general applied by the ignorant patient in disorders totally different from what the quack himself pretends they can cure. Magnificent and unqualified promises inspire weak minds with implicit confidence.
The rise and fall of the success of the tractors were to inspire the poet, Lord Byron,
What varied wonders tempt us as they pass!
The Cow-Pox, Tractors, Galvanism and Gas,
In turns appear, to make the vulgar stare,
Til the swoll’n bubble bursts – and all is air.
Within a decade or so, the bubble surrounding the tractors of Perkins had died out. New forms of quackery were coming in vogue, notably homeopathy and early forms of acupuncture (a practice not dissimilar from tractorism, but relying on medical orientalism rather than galvanism for its inspiration of mechanism.)
Byron obviously has similar difficulties as most of the time in recognising true medical breakthroughs from the quackery that was rife. Vaccines (cow-pox) and anaesthetics (gas) are now fundamental parts of modern practice having been demonstrated to be clearly beneficial. Tractors and galvanism are now unknown outside of history books.
What is remarkable though is how these ideas, that we can be tricked by our expectations, and fooled by illusory cures, still have failed to permeate into some corners of modern medical practice. That the British Journal of General Practice can publish such an obviously flawed paper based on nonsensical and superstitious practices, such as ‘five element acupuncture’, would suggest that something is deeply wrong within the editorial process of that journal.
According to the Economist last month,
Unlike their conventional counterparts, practitioners of alternative medicine often excel at harnessing the placebo effect. They offer long, relaxed consultations with their customers And they believe passionately in their treatments, which are often delivered with great and reassuring ceremony. That alone can be enough to do good, even though the magnets, crystals and ultra-dilute solutions applied to the patients are, by themselves, completely useless.
That is a sentiment that Haygarth would have fully approved off. However, by still believing that the mumbo jumbo really works, quacks are unable to separate out the benefits from a little imagination gone wild from genuine effects on the courses of disease. And whilst such delusions persist, quackery still holds the potential to do much harm.
Thanks for that.
It had totally escaped me that the ‘research’ had been done at the Peninsula College of Medicine and Dentistry at Exeter and Plymouth Universities. Why oh why didn’t they consult Prof Edzard Ernst before they published such absolute nonsense?
Are any GPs raising the probity of the Editor with the Journal?
Nuhn T, Lüdtke R, Geraedts M. Placebo effect sizes in homeopathic compared to conventional drugs – a systematic review of randomised controlled trials. Homeopathy. 2010 Jan;99(1):76-82.
It would be interesting for every treatment and medicine to post a statement as to the evidence for efficacy. For meds it should include comparisons to no treatment and generic, if available. I would like the statement to say something like 5.6 cases occur per 100,000 and with our Brand X there are only 5.1 cases, but there are 5.3 cases with Cheapo, the generic. What would you choose/
I think the information is too limited and too twisted. A consistent pattern of presentation would eventually train people how to read correctly. From the numbers above the .5/.3=167% better is hooey and should be stopped.
Cherry picking and claims printed in mice type size fonts would explode.
I’m a gp and have sent letter to editor.
Please feel free to do the same.
I’ve also asked that the peer reviews be made immediately available and that the paper be withdrawn.
I’m delighted this study has been published. I spent all day Tuesday teaching NHS staff how to critically appraise research articles. When I saw the cover of the BJGP trumpeting an acupuncture study I thought to myself, “I wonder if there’s an error or two in there that I could use to illustrate the points I made during the training session?” And, of course, it’s riddled with mistakes so I’ll be sending it round everyone who attended the training, with a barely suppressed, “I told you so …”
Have a look at this questionable research!