The Quackometer

Labels: Ben Goldacre, chiropractors, edzard ernst, evidence based medicine, Gillian McKeith, quacks

 

Figure1. The Quackometer Quackery Quadrants

Let's call this the Quackometer Quackery Quadrants - of course. How would we divide the scales to use on each axis? For ‘evidence’, this is not too hard. There are accepted measures of the degree of evidence available for a treatment. A heirarchy of medical evidence can be constructed as follows:

1. Systematic Reviews of well controlled Randomized Controlled Trials (meta-analysis) or single RCT with narrow CI (confidence interval)
2. Systematic review cohort studies or lesser quality RCTs
3. Case controlled studies (non randomized)
4. Case series (no control group)
5. Expert opinion (GOBSAT - Good Old Boys Sat Around Table)

Can we construct a similar hierarchy of plausibility? That is possible too. We could, for example, take a mathematical approach and assign the axis a Bayesean prior probability scale. This might be the most desirable approach, but largely impractical in that it is difficult to assign meaningful probabilities to hypotheses, such as the homeopathic one, that 'like-cures-like'. How likely is it that homeopathy will overthrow all that we know about biology? It is vanishingly small, but difficult to be quantitative about it. We can, put a more qualitative scale and grade a treatment according to how well it conforms to well tested knowledge or how much it relies on speculative knowledge or even magical thinking.

1. Proposed mechanism of action based on similar well understood treatments.
2. Consistent with well established biochemistry
3. Consistent with accepted biology and chemistry
4. New biological mechanisms required
5. New chemistry and physics required
6. Inconsistent with accepted physics/chemistry/biology.
7. Requires magical mode of operation/inconsistent with natural laws

You may well come up with your own scale. For the sake of my argument, constructing a definitive and absolute scale is not important. A qualitative approach like the above will do.

So now we have a set of four quadrants that we can use to broadly classify medical interventions according to their plausibility and evidence base. The top right quadrant contains treatments that are well understood in terms of their modes of action and have a good evidence base to support them. The lower left hand quadrant contains interventions that are not based on known science, or rely on pseudoscientific explanations, or even at the extreme magical and supernatural thinking. This is truly the quadrant of quackery.

We would like to think that our medical interventions are all nicely housed in the top right hand quadrant, but this is not the case. For example, the Cochrane methodology, in solely looking at the clinical evidence base will allow us to draw a line of ‘evidence based medicine’ that runs horizontally across the quadrants as shown in Figure 2.

Figure 2. The Realm of Evidence Based Medicine

Everything above the line can be considered as evidence based and, therefore, worthy of public funding and likely to form effective treatments.

However, the problem with this approach can be illustrated with the quackery quadrants. Such a demarcation could possibly allow treatments that have an evidence base, but that are based on highly implausible mechanisms. Can this situation arise? Of course it can.

When medical evidence is evaluated, it is usually of a statistical nature. An arbitrary cut of point is decided where the confidence limits for acceptance becomes defendable. If we get better statistical results than this cut off then we can say we have a significant result. Usually, this cut-off is set at a 95% confidence limit. You may see this written in papers as the p=0.05 threshold. Any test with a p value of less than 0.05 is determined to be of ‘significance’. Unfortunately, the p values in themselves are not enough to tell us if a particular experiment is giving us reliable information about a medical intervention. The p value merely tells us that if the test was fair and unbiased, then what is the probability that the result was merely due to chance and not due to the effects of the intervention? For a p value of 0.05 this means that 1 in 20 fair tests will give the wrong answer.

It is worse than that though as it can be very difficult to construct fair tests. Experiments and reviews can have flawed methodology, incomplete controls and blinding, unpublished results, and, in the worse cases, even be subject to fraud and dishonesty. As such, the proportion of experiments and reviws that give the wrong answer will be much worse than 1 in 20. The upshot of this is that for a highly implausible, but popular alternative medical treatment, then many trials will generate a significant fraction of results that show positive results. If we were to plot the distribution of the various elements of homeopathic evidence on our quackery quadrants, we might end up with something like figure 3.

Figure 3. Where Homeopathic Treatments lie in the Quackery Quadrants

With homeopathy, as we are repeatedly told by the homeopaths, there is an evidence base for supporting the efficacy of their treatments for at least some conditions. This is indeed true, but it is insufficient to convince sceptics that homeopathy is anything other than a placebo. We can see that these positive results, such as the small positive effect in the Oscillococcinum result in the Cochrane review, try to force us to accept that we have a genuine effect from a highly implausible treatment. In other words, we are being forced to accept a miracle. The top left quadrant is indeed the quadrant of miracles in that we are being asked to accept something that appears to be against natural laws.

Now science is not well known for its casual acceptance of miracles, and we should definitely not be accepting the evidence of homeopathic trials as evidence of a medical miracle. The philosopher David Hume (1711-1776) was one of the first to describe the conditions by which we should accept the occurrence of a miracle and that is that the probability that the evidence for the miracle is good evidence should be greater than the probability that the evidence is flawed in some way, such as by mistaken testimony, chance or deceit.

In Hume's words,

When anyone tells me that he saw a dead man restored to life, I immediately consider with myself whether it be more probable, that this person should either deceive or be deceived, or that the fact, which he relates, should really have happened. I weigh the one miracle against the other, and according to the superiority, which I discover, I pronounce my decision, and always reject the greater miracle. If the falsehood of his testimony would be more miraculous than the event which he relates; then, and not till then, can he pretend to command my belief or opinion.

With clinical trials, we have a pretty good idea of what the confidence a trial gives us – typically a 95% confidence level. How confident are we that our basic science of matter is correct? Would you take a 1 in 20 bet that the properties of matter were not to do with atoms? I would suggest that our confidence in basic physics is a lot better than 95% and that homeopathy is in direct contradiction with this knowledge. We have around two hundred years of good research into the properties of matter, collected by thousands of researchers. One little homeopathy study is very unlikely to threaten that body of knowledge. It is much more likely that the positive results of homeopathy are due to statistical chance, poor experimental methodology and even fraud, than showing contradictory evidence for the refutation of fundamental physics.

On our quackery quadrants then, we can draw a line that can tell us when we should accept the result of the evidence before us for any particular treatment. That line will run from the top left to the bottom right. What we are doing here is simply graphically illustrating the mantra of sceptics that extraordinary claims require extraordinary evidence. The corollary to this is that mundane, highly plausible and, dare I say, ‘common sense’ claims require a lower standard of evidence.

Figure 4. The Realm of Scientific Medicine. The evidence base for homeopathy is now excluded from scientific medicine, although may well sit within 'evidence based medicine'

Figure 4 then gives us a quite different view of how to accept the health claims of medicine from the standard one adopted by Cochrane and such bodies as NICE. We are describing scientific medicine as opposed to purely evidence based medicine. Scientific medicine takes into account the scientific context of the evidence and says that we should interpret that evidence in light of what we know about the world. It forbids us from casually accepting light evidence for treatments that are not plausible from what we know about physics, chemistry and biology. We can now only accept the evidence of a treatments efficacy when that evidence is greater then prior probability of that treatment being ineffective. This approach has a number of important implications.

Firstly, and most importantly, to all intents and purposes, clinical trials of highly implausible treatments, such as homeopathy, can never be used as evidence of their efficacy. No matter how good the statistical result of a trial, or how much data is analysed in a meta-analysis, the probability will always be greater that we are just analysing flawed data rather than there being a real effect. Homeopaths complain that sceptics never accept that trial data is proof of the effectiveness of homeopathy. This approach shows that homeopaths are quite right in their fears, although sceptics ought to be careful to point out that it is not because there is no evidence, but rather than the available evidence falls far short of any meaningful threshold of acceptance. Without a degree of plausibility, homeopaths are asking scientists to believe in the daily occurrence of miracles, and that will not do.

This answers my question as to whether Cochrane should be calling for more clinical research. What good would it do if more research was done in Oscillococcinum? More positive results for homeopathy might allow treatments to slip by simplistic ‘evidence based’ criteria for determining effectiveness, but will never satisfy broader scientific scepticism of homeopathy. There is a possible split that exists at the moment where many clinicians working in the NHS provide homeopathy to their patients whilst many academics and scientists are shouting what a nonsense this is. The hospitals are accepting a degree of evidence that is far too weak for real confidence to be expressed in the efficacy of homeopathy. Rather than use a simplistic evidence based approach to deciding which treatments to use in the NHS, a scientific approach needs to be adopted where the prior plausibility of a treatment is first evaluated so that it is possible to decide the degree of evidence required to support that treatment. Not all proposed treatments are the same and can be judged by the same criteria.

By conducting more research, we allow more anomalous evidence to creep in and that can only add to the difficulty of making health care decisions in our hospitals and governments. Rather than clarifying the position, clinical research into highly implausible treatments runs a very high risk of obscuring the truth. It is not that I do not accept that one day a highly implausible treatment will be shown to be effective, but rather there is a far higher chance of producing a nonsense result that just obfuscates the discussion. I will discuss how implausible research should be conducted shortly.

This brings me onto the second point. Homeopaths often accuse sceptics of double standards where low standards of evidence appear to exist for many routine hospital procedures whereas strong evidence is demanded for homeopathy. We can now see that this is not hypocrisy, but an inevitable consequence of scientific thinking. It is perfectly rational to accept treatments as effective if they have very high plausibility but little in the way of good objective evidence. Taking a trivial example, we all know that putting pressure on a wound stops bleeding. But I bet no randomised controlled trials exist to support such a procedure. Would anyone want to doubt that? For many surgical procedures, little in the way of high quality trial data may exist, the evidence may be at worst of the GOBSAT variety. But, many procedures may be inherently less susceptible to biases and subjective measurement errors. Death is a hard measurement point and is not easy to fudge. If a surgical procedure appears to prevent a quick death then we may well be quite right to accept largely anecdotal and case-based evidence. In fact, to insist on randomised controlled trials might well be highly unethical given the high degree of plausibility of the procedure.

This is, of course, in stark contrast to homeopaths claims that their pills can prevent or cure malaria. There is absolutely no good reason to think that this might be true. The plausibility of such a treatment is as near to zero as makes no difference. And yet many homeopaths insist that this is a bedrock of their practice (Hahnemann’s first homeopathic experiments were on malaria). Furthermore, some homeopaths insist on doing their own trials, often in Africa. Such experiments must be totally unethical, because their results, even if positive, could never be sufficient to demonstrate the efficacy of their treatment. Trials such as these put patients at risk with no prospect of any enlightenment to come from that risk.

So, my third point is what sort of research should homeopaths be doing, if any? Well, the only ethical and constructive research that could be done is research that could move homeopathy along the plausibility axis. This would be fundamental research that sought to uncover potential models of how the treatment might work. Before embarking on using real patients as test subject, confidence must be established that a treatment may be effective. That is not just good science but good ethical behaviour.

Figure 5. Direction of Investigations into implausible treatments

Homeopathy has a long path to go along here. Some homeopath supporters recognise this fact and see the importance of both demonstrating their fundamental tenets are true and also trying to show how homeopathy might be integrated into science. (My homeopathy challenge is a simple test to ask homeopaths to demonstrate that their beliefs about the preparation of homeopathic remedies are not just wishful thinking. So far, no one has agreed to the test.) There are some researchers who are looking into so-called ‘memory of water’ effect, that might add a smidgen of plausibility into their claims. So far, the experimental evidence for water memory is woefully inadequate, even if it was in itself a plausible hypothesis.

The utter degree of implausibility is so staggering that I believe it would be difficult to justify public expenditure on fundamental homeopathic research. The only reason it is given any credibility is because so many people have staked their livelihoods in believing it. If Hahnemann had not been born two hundred years ago, but turned up at an NHS hospital today asking them to buy his pills, he would be unceremoniously thrown out for being an utter crank. And that is how we ought to treat homeopaths today.

The news this week has been filled with reports of the relative ineffectiveness of many antidepressant medications. The real shocker is how important data has not been made available to properly establish their effectiveness. Taking this science based medicine approach allows us to clearly differentiate between the different demands of whether more research is warranted into various sorts of antidepressants. Homeopaths may try to seek some equivalence between their failed and partially successful trials and the disappointing evidence for the effectiveness of some antidepressants. Both may look like placebos. But with the conventional pharmaceuticals, plausibility may still be much higher. We may not understand detailed mechanisms for how these drugs affect mood, but at least chemical intervention has some plausibility. My current glass of wine proves that. And, these drugs do show some effect for more depressed people. Understanding why this is and how these effects might be improved would look to be imperative. Homeopathy can make no such claim on limited research money.

And so to summarise, the Cochrane Review should limit its calls for further research to situations where plausible hypotheses exist, as without this, clinical data can never be persuasive. And for sceptics, attacking homeopathy cannot be done by solely by attacking the clinical evidence base. That evidence may well be poor and fragmented but there will always be a constant trickle of positive results such as the Oscillococcinum review, no matter how minor, that allow homeopaths to claim they are part of the evidence based medicine movement and that sceptics are being hypocrites. Homeopathy is wrong because the the evidence that does exist is far too limited for us to accept its efficacy given the extreme implausibility of its action.

****************************************************************************

If you want to explore more of the ideas raised here, a new blog has recently started. ScienceBasedMedicine.org is being written by prominent sceptic bloggers such as Steven Novella, Wallace Sampson, Harriet Hall and David H. Gorski.

Labels: evidence based medicine, homeopathy, pseudoscience

 

Double Blind Randomised Controlled Trials (DBRCTs) are not the preserve of medical science. Increasingly, corporations are using them to enter into a new world of Evidence Based Marketing where massive, continuous and automated trials are being run on us in order find out how to sell more stuff and become more profitable. What does this mean for medicine and our lives?

Our beliefs are our internal models of the outside world that help us inform decisions. When faced with a need for action we must have beliefs about what our goals are, how valuable those goals are, the chances of succeeding in moving towards those goals when taking a particular action and what are the consequences of failure. Our brain is calculating what our 'next best action' is so that we maximise our pay back from the action. As such, having beliefs that correspond to reality helps us make good decisions. But we are not like computers where we assign numeric probabilities to our chances of success, or correctly value our goals, or estimate the consequences of failure. We most often have intuitive feels for our next best actions that may be hard to express to another person. If we become experts in a certain field though, those intuitions may be more commonly replaced with learned knowledge and we may make more conscious and explicit calculations. But completely ignoring our intuition is hard.

Doctors are faced with a constant stream of choices. At each stage of their interactions with a patient there are dozens of decisions and options that need to be analysed, prioritised and then made. Typically, only one thing can be done at a time. What is the best thing to do next with this patient given what they have told me, what diagnostic information I have available, their history and my experience? Should I wait to see how things progress and get more information? Is surgery the right option or is medication right? Which medication? Should I refer to a more specialist colleague? What risks are there in the various courses of action? What is the patient's attitude to these risks? What impact would a poor outcome have on the patient? This is complex stuff and the training required to get to the state where you can start making such decisions on your own is likely to be over a decade – maybe two. Getting it wrong can have severe repercussions.

The big change over the last few decades in medicine is that, increasingly, historical statistical evidence is being used to help inform such decisions about 'next best actions'. Before the advent of the Evidence Based Medicine movement, such decisions were largely made on experience, training and intuition. The problems of this approach were that any particular doctor can only ever achieve a certain stock of experience and training. In a career, the number of times any doctor might have seen a certain rare condition could be counted on the fingers of one hand, if at all. Even for common conditions, a doctor’s experience of various modes of treatment may well be limited. What is worse is that personal intuition and experience are subject to horrible biases, cognitive errors and delusions that make judgements very suspect. Evidence Based Medicine has added new sources of information into the decision making process in the form of pooled data from thousands of patients. We can now assign numerical probabilities to our chances of success given data on what has happened in the past. Doctors have slowly subjugated their personal intuition and experience to the collective authority of the evidence from thousands of carefully collated cases, often in the form of clinical trials.

When faced with the question ‘What is the next best action with this patient?’ a doctor can draw on the statistical evidence of thousands of cases somewhat like the one in front of her. Double-blind randomised controlled trials (DBRCTs) are now the benchmark method for providing the statistical evidence to help inform good choices. So now, an understanding of interpreting statistical data is as much a requirement of doctors as a detailed knowledge of anatomy. This change has not been smooth and is still ongoing as this development has sometimes been seen as a threat to the authority and power of the individual consultant.

Also, the medical problem of collecting the data to allow accurate predictions of outcomes can be very difficult. It is expensive, requires lots of ethical considerations and consent from many participants. There is then the difficult problem of ensuring the results can be turned into effective actions by ensuring that medical staff are made aware of the results and that they have access to these results at their point of need.

For the patients and the public, statistical evidence can be treated with suspicion. The adage that there are 'lies, damn lies, and statistics' has come to be interpreted as that 'all statistics lie', rather than 'statistics can be misused to deceive'. But, what is pretty much unknown is how deep statistical evidence is being used to affect our lives. It is not just medicine that has noticed the power of obtaining data from controlled trials in order to make better decisions. Where this sort of thinking is increasingly making the biggest impact is not in hospitals but in a marketing department near you. And they are using these techniques to the most devastating effect. There have been no big breakthroughs in mathematics that is making this happen. Rather, software is becoming cheap and easily deployable and usable by people without PhDs in statistics. Web technology allows the intelligence discovered through statistical analysis to be easily delivered and used by the companies employees and on automated web sites. Marketing departments are using sophisticated software without being aware of its underlying complexity.

Let’s look at a mobile phone company. On the face of it, a simple business. They sell you a mobile phone and a tariff and collect some money off you monthly, or through some sort of voucher scheme. However, it is now a hugely competitive business where anyone who is ever likely to own a phone probably now does (or maybe more than one). The fight is for the customer loyalty of the most profitable customers and to sell more services to them. How does a mobile phone company do this?

Imagine that you bought a mobile phone a year ago. You are now at the end of your contract and the flashy phone you acquired last year now resembles a house brick and is as trendy as acne. You are free to stop paying your line rental and jump to a new company or get an upgrade with your existing company. What should the mobile phone company do? They could email you and offer you a new, smart, sexy phone and good deal on a tariff. But, this might alert you to the fact that you are free to jump ship and get that iPhone you know you need from a competitor. Alternatively, your company could just keep quiet and hope you keep paying your bills each month at no cost to them. Upgrading your phone is expensive for the company (in the UK, handsets are heavily subsidised). Your company does not want to do this if they do not have to. But, if they do nothing, you might quietly slip away without them ever knowing. Should they contact you or not?

The answer is statistical analysis of what other customers like you have done in the past – predictive analytics. Given your age, your address, your monthly spend, the number of texts you send, your overseas calls, the amount of time you chat on the phone, your payment history, your payment method, the complaints you have made, the ring tones you download and your eye-colour, your mobile phone company can make a good guess at your likelihood to ‘churn’ – that is, up sticks and go to the competition. Moreover, it can predict how much you might spend next year and so work out how much it can afford to spend trying to keep you as a customer (or if it should quietly ‘let you go’.) Some customers get offered low tariffs, flashy gold plated handsets, text bundles and free insurance. Others get offered a free ringtone. Statistical analysis is deciding what you get.

But it gets more sophisticated than this. If you do happen to ring up your mobile phone call centre, the complete experience, second by second, will be decided by propensity analyses to decide the ‘Next Best Action’ at every stage of the call. Do you get directed to a real agent or put in the queue for the dreaded speaking machine? Which agent will speak to you? How old will the agent be? Will they support the same football team or like the same soaps? What will that agent say to you? Every sentence the agent says will have a computer suggesting the next best thing to say. Do they offer you a phone upgrade? Ask you to pay by Direct Debit? Offer you a deal on home broadband? Agents may not have to ask you these things, but they will quickly learn their bonuses improve if they do. Computers are whirring away in the background constantly re-evaluating what is the next best thing to do. If you happen to owe your mobile phone company money on a bill, you can bet you will have been sent to one of their specialist 'collections' agents who will be following strategies suggested by the computer to get you to promise to pay.

Frequently, sufficient historical data is not available to make accurate predictions about how you will behave. In comes real-time adaptive controlled trials. When the companies next best actions are being enacted, your responses are being collected and recorded. In the background, software is running an experiment to work out what to do in the future. The software creates experiments, assigns strategies, and randomly splits the customer base into experimental groups and control groups without any human being aware who is in what group. The software is counting response types in real time and adjusts its next predictions in response to what is happening right now. A competitor might bring out a new special offer. The software recognises that certain customer ‘treatments’ are no longer quite so effective and so quietly drops certain offers, or reserves them for the most ‘price insensitive’ customers. All this could happen within hours and without a real marketeer lifting a finger. The management are no longer deciding how to market what to which customers. They are simply defining corporate strategies, such as their profitability goals or the extent of their customer base, and the statistical analysis software is quietly getting on with the job of deciding who to offer which products and when to do it. Call centre agents are none the wiser that the strategies they are using on their callers are slowly and continuously changing as the evidence base for their effectiveness is evolving.

And it is not just the call centre – the adverts that appear on your tailored personal banking web site, the leaflets that go in your statement and the promotional text messages you receive are all part of a unified and optimised strategy that is unique to you based on the vast amount of data that the company has on you, and, moreover, it is based on the evidence of effectiveness of hundreds of thousands of previous interactions with similar customers. Most often, a company will decide that their next best action is to do nothing; they know that bombarding you with silly sales messages just annoys you. The 'Old Company' sends you a hundred leaflets knowing that less than 1% will hit a receptive target and that the rest are actually doing the company harm. But without predictive capability, it has no choice. The trick is to know when that sales message will hit a chord. The beliefs that inform companies when is the best time to sell to you are no longer held in the intuitive minds of the marketing department but in the coefficients of predictive models in a computer.

Is this happening right now? It is getting very close. Scenarios like these are happening in the majority of large consumer companies. They are learning how to use the various strands of these technologies and deploying more and more. John Wanamaker, the department store owner famously said, "Half the money I spend on advertising is wasted; the trouble is I don't know which half." If he was only wasting half, then he was doing well. We are now witnessing the emergence of Evidence Based Marketing where Wanamaker's maxim will be laid to rest. Companies are talking about Enterprise Decision Management where the technologies required to do this are brought together into a central decision hub that is used to ensure the myriad of tiny and large decisions that are being made day-to-day are manageable, understandable, controllable and based on evolving evidence.

The most aggressive mobile phone companies are starting to behave just like this example, and others are following suit. It is not just phone companies, but your cable TV company, your bank and your electricity supplier. Any large company not doing this in five to ten years time will not be a company. Already, some of the largest banks are making billions of next best action decisions per year using this sort of strategy. It is not just the big decisions, such as whether to lend you money, but thousands of tiny decisions such as to ask you whether the company has your correct home phone number or your cable company to remind you that a blockbuster film you might enjoy is on tonight. Your total commercial experience will be determined by the results of thousands of double-blind randomised controlled trials, and you will be taking part in a dozen more trials, right now, without you even knowing. We are all now consumer lab rats.

For the consumer, the experience may not always be positive. Some may feel that their company appears to ‘know what they want’. Others may feel locked out of good deals. We are allowing this technology into our lives without us really thinking about it. Amazon tells you what is the next best book to buy and what music you will like. Google displays adverts that have trial-based evidence behind them to suggest you might click on them. Your Internet dating site is telling you who your next best lover is. The songs you listen to, the books you buy and the news you read is all personally decided on a ‘demographic of one’ and may well be unique to your own tastes, buying habits, politics and preferences, and all decided by statistical analyses of huge numbers of people. Sophisticated customers know this and play this to their advantage. If you are not threatening to leave your mobile phone company next time your contract is up you will not be passed to their ‘retention team’ and so will be denied the best deals. Complain too much though, and your mobile phone company (or, actually the underlying software) will decide you are not worth the bother. It’s going to be a battle. Expect consumers to retaliate, or at least, expect other start-up companies that use even more statistical evidence to help you retaliate and get the best deals. When should you buy that easyJet ticket or book that holiday? Do you know when it will be the cheapest without risking that it will sell out? Expect someone to help you soon. The First Consumer Statistics War is beginning.

Our reaction to this technology might be horror. It is impersonal and diabolical. Its advocates claim that it is just making large companies more like your local butcher who knows your name, knows you like smokey rindless bacon and you are good for credit. The reality is that it is a bit of both: a dehumanisation that makes the experience somewhat more personal. It really is just a massive extension of what your butcher does: he is nice to you and knows you well, so that he get the most out of you without you feeling cheated or exploited and start shopping elsewhere. Both you and your butcher feel happy with this arrangement.

How will this affect the relationship you have with your doctor? Well, I doubt it will be quite so brutal or extensive - just yet. The main difference is that it is not solely the doctor deciding what the desired outcomes are. The key word is 'best' in Next Best Action. Who says what is best? In corporations, they set their own goals depending on corporate strategy: number of customers, profit per customer, quarterly sales targets, etc. You have no say. With the doctor, their emerging role is to help you decide where you want to end up given the various risks and benefits of various courses of treatment. Would surgery risk curtailing your mobility, hobbies and family life? Would it be better to look at other therapies right now? Part of the doctor’s job is to help patients understand the statistical nature of what might happen. As the homeopaths are so fond of telling us, treatments must be individualised, but it is evidence based medicine that can really help individually tailor those paths of action using a good understanding of the risks and benefits of each path and your personal goals.

As I said earlier, one of the difficult problems of evidence based medicine is turning the currently available evidence into actions. It is a real problem to get the data to the doctors at the right time and the right place and in a form that can be used to help make informed decisions. Maybe doctors may learn something from their ruthless commercial colleagues. It may never happen they we see doctors simply following scripts from a computer (with the possible exception of hugely complex and time-pressured emergency and intensive care), but computers will be at hand to provide the most up to date evidence base for various courses of action. Unlike commercial data, the collation of vast amounts of health data to help in this process may not be easy. Even collecting anonymised data to help produce rolling ‘real time experiments’ is fraught with ethical and technological difficulties. The rewards are real. Statistical technology has the potential to offer us better and faster diagnosis, clearer understanding of treatment options and higher chances of favourable outcomes.

Medicine is undoubtedly moving in this direction and our reaction to it will be interesting. Already, dissatisfaction with a doctor is one of the key reasons why people turn to alternative medicine. Will the perception of increased impersonality made by the presence of computer aided consultations push people further into the arms of quacks? Will the irony of improved diagnosis and treatment plans risk people seeking unproven and dangerous alternatives?

Whatever happens, I think the emerging and defining difference between conventional health care and quackery will be the role of statistical data in helping to determine the next best actions for patients. Nowhere is this starker than how homeopaths deal with evidence in their practice. Homeopathy, if it survives, will become a living museum of what medicine used to be like before statistical data was used to help determine actions. Paternal and ineffective, but personable. Homeopaths use their ‘experience’, their 'training', their intuition, and their bible – the Organon. No homeopath ever uses the statistical data from a trial to determine which remedy to prescribe. What homeopaths end up using is their delusions and wishful thinking. In fact, the resistance to evidence in homeopathy is overwhelming and defining. For someone wishing to maintain their health, their next best action might be to ignore those not using statistics to decide their next best action.

Labels: evidence based medicine