The Quackometer started as an experiment to see if it is easy to spot quack web sites just from the language they use.
Several people have noted how quack web-sites use similar language and vocabulary and once you can spot the patterns, spotting quackery is easy. Quack words include “energy”, “holistic”, “vibrations”, “magnetic healing”, “quantum” . These words are usually borrowed from physics and used to promote dubious health claims. As such, their use is pseudoscientific and just meant to impress and bamboozle the gullible.
So, is it possible to spot a quack web site just from its use of language? Is is possible to automate the process? The Quackometer intends to find out…
If this works, then all the public need do when faced with suspicious claims, is put the suspect URL into this web site and my little friend, the black duck, will analyse the page and give a verdict.
My name is Andy and I am the inventor of the quackometer. I also write the blog.
The Quackometer and blog are experiments in critical thinking. If you read what I am writing, I very rarely venture past using a basic understanding of science. Most of the time, what I am doing is spotting common errors of thinking and argument, such as post hoc thinking, magical thinking, selective thinking and appeals to authority. As such, examining the claims of alleged quacks rarely relies on detailed medical knowledge. As such, being critical of health claims rarely needs detailed medical knowledge and, as I hope this is something we can all take part in and debate.
A common response to my posts has been to question my qualifications for writing. This is known as an ad hominem attack and I will always try not to engage. The truth of whether homeopathy is better than a placebo has nothing to with what exams I have sat, and how many hours I have studied homeopathy texts. Either it works or it doesn’t. Our food today is either OK to eat or worryingly nutrient poor. The certificates on my study wall have nothing to do with this. These days, we all have access to vast amounts of information on the web. I want to debate what is good evidence and what is rubbish. I want to see who is presenting good arguments for their claims and who is talking gobbledegook.
For this reason, I do not want to offer chances for my critics to start fights about my education. I want to stick to the arguments. I am not trying to be anonymous. You can contact me whenever you like and I will gladly engage with you, but lets focus on the arguments rather than personal details.
Oh, and the usual disclaimer. This is a site about critical thinking – it is not giving medical advice. Go and see your doctor if something is worrying you.
Who Funds This Site?
If I had a pound for every quack who has accused me of being in the pay of ‘Big Pharma’ I would be richer than if I was really in their pay.
It looks like accusing someone of just being a shill for evil medical interests is a standard way that quacks avoid answering the criticisms made against them. Its very shallow. An experiment can be good and a scientific conclusion sound even if someone is receiving money from someone you don’t like. I always try to criticise the science first – and then look for potential sources of conflict of interest if the science is found to be wanting. Quacks often do it the other way around. They use potential sources of conflict of interest as excuses for ignoring the evidence. Ignoring the science and just calling ‘shill’ is tantamount to calling someone a fraud – but in a cowards way.
So, for the record. This site costs less than a hundred quid per year to run. I do stuff in my own time and do not rely on contributions. I do not solicit contributions. I will be opening a quackometer shop to sell the odd mug or t-shirt. Hopefully, this will cover my hosting costs. Any excess will be used to buy dodgy quack products or services in order to show how real they are.
What is Quackery?
Definitions are hard. You could argue that one person’s quack is another’s health professional. I do not want to limit the definition to just those people who practice Complementary and Alternative Medicine (CAM). Your local GP may, on occaisions, resort to quackery. Also, a CAM practitioner may be very diligent in how they present themselves. I shall take a working definition from the excellent Quackwatch web site. This definition appears to be quite neutral as to what sort of person is the source of the quackery…
Quackery, in the broadest terms, is “anything involving overpromotion in the field of health.”
A quack is “a pretender to medical skill; a charlatan”
and “one who talks pretentiously without sound knowledge of the subject discussed.”
(This is is essential to the Quackometer Project as spotting quackery depends on spotting this pretentious, out-of-context vocabulary.)
Quackery is often, but not always, linked to health fraud where there is “the promotion, for profit, of a medical remedy known to be false or unproven.”
How can you spot Quack? – they nearly always do the following:-
- Flaunted qualifications and credentials – this is just an ‘appeal to authority’. Quacks often award themselves impressive qualifications or buy them from non-accredited ‘colleges’ usually in he USA.
- Exagerated and inflated claims – diets, cures or remedies appear to solve a whole host of illnesses and problems, not just one problem – they are non-specific. Foods are not just foods, but ‘superfoods’ etc.
- More often seen on TV, newspapers, magazines with their ‘latest findings’ than in scientific journals, conferences, text books.
- Works alone – a sole genius in a world that won’t listen.
- Use of out-of-context language, e.g. energy, frequencies, vibrations, biomagnetic, quantum, detoxification, organic, holistic… These words are often stolen from other disciplines (usually physics) with the quack having no idea what they mean. Their use in health matters is pseudoscience and meant to sound impressive and to bamboozle the gullible.
- Lots of impressive testimonials – little or no independent peer-reviewed research, no ballance in reviews of research, i.e. no mention of negative results, untracable privately published ‘research’, lots of ‘happy customers’. Testimonials count for nothing – anyone can get them for anything. People fool themselves over the effectiveness of treatments.
- Claims to be standing up for ordinary people against the conspiracy of ‘big pharma’, doctors, scientist, the government, multinationals and other great evils (who might disagree with them).
- Say there is always a need for a personalised questionaire, consultation, membership (with them, not your GP) – just a way to flog more rubbish.
…and much more, Maybe I can add to this list in my blog as time goes by.
Are Quacks Fraudsters?
In fact, I would sat that most people I have come across are not dishonest in the sense that they are knowingly ripping people off. Most quacks I discuss here genuinely believe what they are are talking about and think they are helping people. As such they are ‘good’ people.
What most quacks are guilty off, if anything, is a certain carelessness with the truth. They are delusional in that they hold false beliefs. Are they to blame for this? Not necessarily. We all hold delusional beliefs about one thing or another. Where culpability may start to creep in is when obvious canards get repeated in the face of them being shown to be false defenses. Homeopathists appear to be particularly good at this.
However, there are some real thieving gits out there too. People who knowingly are ripping people off. I will try to make this clear when I believe this to be true. However, motives are always are hard thing to fathom.
In he middle, is the quack who does not care about the truth. These are the Quack Bullshitters.
It is impossible for someone to lie unless he thinks he knows the truth. Producing bullshit requires no such conviction. A person who lies is thereby responding to the truth, and he is to that extent respectful of it. When an honest man speaks, he says only what he believes to be true; and for the liar, it is correspondingly indispensable that he considers his statements to be false. For the bullshitter, however, all these bets are off: he is neither on the side of the true nor on the side of the false. His eye is not on the facts at all, as the eyes of the honest man and of the liar are, except insofar as they may be pertinent to his interest in getting away with what he says. He does not care whether the things he says describe reality correctly. He just picks them out, or makes them up, to suit his purpose.
The Science of Quackometrics
The quackometer counts words in web pages that quacks tend to use. The more quack words, the more quackery is suspected. That is Quackometrics.
The basic problem is that spotting the suspect words that many sites use, such as ‘vibrations’ or ‘energy’ is just not good enough as ‘good science’ sites are quite at liberty to use them. Even spotting these words in close conjunction with health terms, such as ‘healing’ or ‘nutrients’, is not quite good enough. My own background was research within in nuclear medicine group and the researchers had lots of legitimate reasons to mention ‘magnets’ and ‘health’ in (almost) the same breath.
So – the site uses an algorithm roughly like this:
Keep a number of different dictionaries for use in tallying words in a web site
Load the suspect web page and strip as much out as possible, HTML tags, scripts, punctuation etc.
Count the number of words in each of the following dictionaries:
a) altmed terms: such as ‘homeopathic’, ‘herbal’, ‘naturopath’
b) pseudoscientific: clearly suspect terms that scientists rarely use such as ‘toxins’, ‘superfoods’.
c) domain specific words from biomed, physics or chemistry such as ‘energy’, ‘vibration’, ‘organic’.
d) skeptical words: words that no sincere homeopath would ever use, such as ‘placebo’, ‘flawed’, ‘crank’ or ‘prosecution’.
e) commerce terms that would indicate that something is for sale, such as ‘products’, ‘shipping’, or ‘p&p’.
f) Run a few other checks on pomo terms and religious terms, although not much is done with these.
Compare the ratio of frequency usage of these various types of terms and compare them to preset thresholds. If a threshold is exceeded then append the test’s associate sentence to the response. The tweaking I have been doing to the site has been adding words to dictionaries and varying the thresholds for matches.
This does not always work, Some quacks are very clever and avoid the obvious quack words. Nonetheless they still have completely hatstand ideas.
So, if anyone else has suggestions, then I would be very grateful. Just need to give up my real job to concentrate on this now.