You are Taking Part in a Randomised Controlled Trial Right Now
Double Blind Randomised Controlled Trials (DBRCTs) are not the preserve of medical science. Increasingly, corporations are using them to enter into a new world of Evidence Based Marketing where massive, continuous and automated trials are being run on us in order find out how to sell more stuff and become more profitable. What does this mean for medicine and our lives?
Our beliefs are our internal models of the outside world that help us inform decisions. When faced with a need for action we must have beliefs about what our goals are, how valuable those goals are, the chances of succeeding in moving towards those goals when taking a particular action and what are the consequences of failure. Our brain is calculating what our ‘next best action’ is so that we maximise our pay back from the action. As such, having beliefs that correspond to reality helps us make good decisions. But we are not like computers where we assign numeric probabilities to our chances of success, or correctly value our goals, or estimate the consequences of failure. We most often have intuitive feels for our next best actions that may be hard to express to another person. If we become experts in a certain field though, those intuitions may be more commonly replaced with learned knowledge and we may make more conscious and explicit calculations. But completely ignoring our intuition is hard.
Doctors are faced with a constant stream of choices. At each stage of their interactions with a patient there are dozens of decisions and options that need to be analysed, prioritised and then made. Typically, only one thing can be done at a time. What is the best thing to do next with this patient given what they have told me, what diagnostic information I have available, their history and my experience? Should I wait to see how things progress and get more information? Is surgery the right option or is medication right? Which medication? Should I refer to a more specialist colleague? What risks are there in the various courses of action? What is the patient’s attitude to these risks? What impact would a poor outcome have on the patient? This is complex stuff and the training required to get to the state where you can start making such decisions on your own is likely to be over a decade – maybe two. Getting it wrong can have severe repercussions.
The big change over the last few decades in medicine is that, increasingly, historical statistical evidence is being used to help inform such decisions about ‘next best actions’. Before the advent of the Evidence Based Medicine movement, such decisions were largely made on experience, training and intuition. The problems of this approach were that any particular doctor can only ever achieve a certain stock of experience and training. In a career, the number of times any doctor might have seen a certain rare condition could be counted on the fingers of one hand, if at all. Even for common conditions, a doctor’s experience of various modes of treatment may well be limited. What is worse is that personal intuition and experience are subject to horrible biases, cognitive errors and delusions that make judgements very suspect. Evidence Based Medicine has added new sources of information into the decision making process in the form of pooled data from thousands of patients. We can now assign numerical probabilities to our chances of success given data on what has happened in the past. Doctors have slowly subjugated their personal intuition and experience to the collective authority of the evidence from thousands of carefully collated cases, often in the form of clinical trials.
When faced with the question ‘What is the next best action with this patient?’ a doctor can draw on the statistical evidence of thousands of cases somewhat like the one in front of her. Double-blind randomised controlled trials (DBRCTs) are now the benchmark method for providing the statistical evidence to help inform good choices. So now, an understanding of interpreting statistical data is as much a requirement of doctors as a detailed knowledge of anatomy. This change has not been smooth and is still ongoing as this development has sometimes been seen as a threat to the authority and power of the individual consultant.
Also, the medical problem of collecting the data to allow accurate predictions of outcomes can be very difficult. It is expensive, requires lots of ethical considerations and consent from many participants. There is then the difficult problem of ensuring the results can be turned into effective actions by ensuring that medical staff are made aware of the results and that they have access to these results at their point of need.
For the patients and the public, statistical evidence can be treated with suspicion. The adage that there are ‘lies, damn lies, and statistics’ has come to be interpreted as that ‘all statistics lie’, rather than ‘statistics can be misused to deceive’. But, what is pretty much unknown is how deep statistical evidence is being used to affect our lives. It is not just medicine that has noticed the power of obtaining data from controlled trials in order to make better decisions. Where this sort of thinking is increasingly making the biggest impact is not in hospitals but in a marketing department near you. And they are using these techniques to the most devastating effect. There have been no big breakthroughs in mathematics that is making this happen. Rather, software is becoming cheap and easily deployable and usable by people without PhDs in statistics. Web technology allows the intelligence discovered through statistical analysis to be easily delivered and used by the companies employees and on automated web sites. Marketing departments are using sophisticated software without being aware of its underlying complexity.
Let’s look at a mobile phone company. On the face of it, a simple business. They sell you a mobile phone and a tariff and collect some money off you monthly, or through some sort of voucher scheme. However, it is now a hugely competitive business where anyone who is ever likely to own a phone probably now does (or maybe more than one). The fight is for the customer loyalty of the most profitable customers and to sell more services to them. How does a mobile phone company do this?
Imagine that you bought a mobile phone a year ago. You are now at the end of your contract and the flashy phone you acquired last year now resembles a house brick and is as trendy as acne. You are free to stop paying your line rental and jump to a new company or get an upgrade with your existing company. What should the mobile phone company do? They could email you and offer you a new, smart, sexy phone and good deal on a tariff. But, this might alert you to the fact that you are free to jump ship and get that iPhone you know you need from a competitor. Alternatively, your company could just keep quiet and hope you keep paying your bills each month at no cost to them. Upgrading your phone is expensive for the company (in the UK, handsets are heavily subsidised). Your company does not want to do this if they do not have to. But, if they do nothing, you might quietly slip away without them ever knowing. Should they contact you or not?
The answer is statistical analysis of what other customers like you have done in the past – predictive analytics. Given your age, your address, your monthly spend, the number of texts you send, your overseas calls, the amount of time you chat on the phone, your payment history, your payment method, the complaints you have made, the ring tones you download and your eye-colour, your mobile phone company can make a good guess at your likelihood to ‘churn’ – that is, up sticks and go to the competition. Moreover, it can predict how much you might spend next year and so work out how much it can afford to spend trying to keep you as a customer (or if it should quietly ‘let you go’.) Some customers get offered low tariffs, flashy gold plated handsets, text bundles and free insurance. Others get offered a free ringtone. Statistical analysis is deciding what you get.
But it gets more sophisticated than this. If you do happen to ring up your mobile phone call centre, the complete experience, second by second, will be decided by propensity analyses to decide the ‘Next Best Action’ at every stage of the call. Do you get directed to a real agent or put in the queue for the dreaded speaking machine? Which agent will speak to you? How old will the agent be? Will they support the same football team or like the same soaps? What will that agent say to you? Every sentence the agent says will have a computer suggesting the next best thing to say. Do they offer you a phone upgrade? Ask you to pay by Direct Debit? Offer you a deal on home broadband? Agents may not have to ask you these things, but they will quickly learn their bonuses improve if they do. Computers are whirring away in the background constantly re-evaluating what is the next best thing to do. If you happen to owe your mobile phone company money on a bill, you can bet you will have been sent to one of their specialist ‘collections’ agents who will be following strategies suggested by the computer to get you to promise to pay.
Frequently, sufficient historical data is not available to make accurate predictions about how you will behave. In comes real-time adaptive controlled trials. When the companies next best actions are being enacted, your responses are being collected and recorded. In the background, software is running an experiment to work out what to do in the future. The software creates experiments, assigns strategies, and randomly splits the customer base into experimental groups and control groups without any human being aware who is in what group. The software is counting response types in real time and adjusts its next predictions in response to what is happening right now. A competitor might bring out a new special offer. The software recognises that certain customer ‘treatments’ are no longer quite so effective and so quietly drops certain offers, or reserves them for the most ‘price insensitive’ customers. All this could happen within hours and without a real marketeer lifting a finger. The management are no longer deciding how to market what to which customers. They are simply defining corporate strategies, such as their profitability goals or the extent of their customer base, and the statistical analysis software is quietly getting on with the job of deciding who to offer which products and when to do it. Call centre agents are none the wiser that the strategies they are using on their callers are slowly and continuously changing as the evidence base for their effectiveness is evolving.
And it is not just the call centre – the adverts that appear on your tailored personal banking web site, the leaflets that go in your statement and the promotional text messages you receive are all part of a unified and optimised strategy that is unique to you based on the vast amount of data that the company has on you, and, moreover, it is based on the evidence of effectiveness of hundreds of thousands of previous interactions with similar customers. Most often, a company will decide that their next best action is to do nothing; they know that bombarding you with silly sales messages just annoys you. The ‘Old Company’ sends you a hundred leaflets knowing that less than 1% will hit a receptive target and that the rest are actually doing the company harm. But without predictive capability, it has no choice. The trick is to know when that sales message will hit a chord. The beliefs that inform companies when is the best time to sell to you are no longer held in the intuitive minds of the marketing department but in the coefficients of predictive models in a computer.
Is this happening right now? It is getting very close. Scenarios like these are happening in the majority of large consumer companies. They are learning how to use the various strands of these technologies and deploying more and more. John Wanamaker, the department store owner famously said, “Half the money I spend on advertising is wasted; the trouble is I don’t know which half.” If he was only wasting half, then he was doing well. We are now witnessing the emergence of Evidence Based Marketing where Wanamaker’s maxim will be laid to rest. Companies are talking about Enterprise Decision Management where the technologies required to do this are brought together into a central decision hub that is used to ensure the myriad of tiny and large decisions that are being made day-to-day are manageable, understandable, controllable and based on evolving evidence.
The most aggressive mobile phone companies are starting to behave just like this example, and others are following suit. It is not just phone companies, but your cable TV company, your bank and your electricity supplier. Any large company not doing this in five to ten years time will not be a company. Already, some of the largest banks are making billions of next best action decisions per year using this sort of strategy. It is not just the big decisions, such as whether to lend you money, but thousands of tiny decisions such as to ask you whether the company has your correct home phone number or your cable company to remind you that a blockbuster film you might enjoy is on tonight. Your total commercial experience will be determined by the results of thousands of double-blind randomised controlled trials, and you will be taking part in a dozen more trials, right now, without you even knowing. We are all now consumer lab rats.
For the consumer, the experience may not always be positive. Some may feel that their company appears to ‘know what they want’. Others may feel locked out of good deals. We are allowing this technology into our lives without us really thinking about it. Amazon tells you what is the next best book to buy and what music you will like. Google displays adverts that have trial-based evidence behind them to suggest you might click on them. Your Internet dating site is telling you who your next best lover is. The songs you listen to, the books you buy and the news you read is all personally decided on a ‘demographic of one’ and may well be unique to your own tastes, buying habits, politics and preferences, and all decided by statistical analyses of huge numbers of people. Sophisticated customers know this and play this to their advantage. If you are not threatening to leave your mobile phone company next time your contract is up you will not be passed to their ‘retention team’ and so will be denied the best deals. Complain too much though, and your mobile phone company (or, actually the underlying software) will decide you are not worth the bother. It’s going to be a battle. Expect consumers to retaliate, or at least, expect other start-up companies that use even more statistical evidence to help you retaliate and get the best deals. When should you buy that easyJet ticket or book that holiday? Do you know when it will be the cheapest without risking that it will sell out? Expect someone to help you soon. The First Consumer Statistics War is beginning.
Our reaction to this technology might be horror. It is impersonal and diabolical. Its advocates claim that it is just making large companies more like your local butcher who knows your name, knows you like smokey rindless bacon and you are good for credit. The reality is that it is a bit of both: a dehumanisation that makes the experience somewhat more personal. It really is just a massive extension of what your butcher does: he is nice to you and knows you well, so that he get the most out of you without you feeling cheated or exploited and start shopping elsewhere. Both you and your butcher feel happy with this arrangement.
How will this affect the relationship you have with your doctor? Well, I doubt it will be quite so brutal or extensive – just yet. The main difference is that it is not solely the doctor deciding what the desired outcomes are. The key word is ‘best’ in Next Best Action. Who says what is best? In corporations, they set their own goals depending on corporate strategy: number of customers, profit per customer, quarterly sales targets, etc. You have no say. With the doctor, their emerging role is to help you decide where you want to end up given the various risks and benefits of various courses of treatment. Would surgery risk curtailing your mobility, hobbies and family life? Would it be better to look at other therapies right now? Part of the doctor’s job is to help patients understand the statistical nature of what might happen. As the homeopaths are so fond of telling us, treatments must be individualised, but it is evidence based medicine that can really help individually tailor those paths of action using a good understanding of the risks and benefits of each path and your personal goals.
As I said earlier, one of the difficult problems of evidence based medicine is turning the currently available evidence into actions. It is a real problem to get the data to the doctors at the right time and the right place and in a form that can be used to help make informed decisions. Maybe doctors may learn something from their ruthless commercial colleagues. It may never happen they we see doctors simply following scripts from a computer (with the possible exception of hugely complex and time-pressured emergency and intensive care), but computers will be at hand to provide the most up to date evidence base for various courses of action. Unlike commercial data, the collation of vast amounts of health data to help in this process may not be easy. Even collecting anonymised data to help produce rolling ‘real time experiments’ is fraught with ethical and technological difficulties. The rewards are real. Statistical technology has the potential to offer us better and faster diagnosis, clearer understanding of treatment options and higher chances of favourable outcomes.
Medicine is undoubtedly moving in this direction and our reaction to it will be interesting. Already, dissatisfaction with a doctor is one of the key reasons why people turn to alternative medicine. Will the perception of increased impersonality made by the presence of computer aided consultations push people further into the arms of quacks? Will the irony of improved diagnosis and treatment plans risk people seeking unproven and dangerous alternatives?
Whatever happens, I think the emerging and defining difference between conventional health care and quackery will be the role of statistical data in helping to determine the next best actions for patients. Nowhere is this starker than how homeopaths deal with evidence in their practice. Homeopathy, if it survives, will become a living museum of what medicine used to be like before statistical data was used to help determine actions. Paternal and ineffective, but personable. Homeopaths use their ‘experience’, their ‘training’, their intuition, and their bible – the Organon. No homeopath ever uses the statistical data from a trial to determine which remedy to prescribe. What homeopaths end up using is their delusions and wishful thinking. In fact, the resistance to evidence in homeopathy is overwhelming and defining. For someone wishing to maintain their health, their next best action might be to ignore those not using statistics to decide their next best action.
30 comments