So, how does the Quackometer work?
The quackometer counts words in web pages that quacks tend to use. The more quack words, the more quackery is suspected. That is Quackometrics.
The basic problem is that spotting the suspect words that many sites use, such as ‘vibrations’ or ‘energy’ is just not good enough as ‘good science’ sites are quite at liberty to use them. Even spotting these words in close conjunction with health terms, such as ‘healing’ or ‘nutrients’, is not quite good enough. My own background was research within in nuclear medicine group and the researchers had lots of legitimate reasons to mention ‘magnets’ and ‘health’ in (almost) the same breath.
So – the site uses an algorithm roughly like this:
- Keep a number of different dictionaries for use in tallying words in a web site
- Load the suspect web page and strip as much out as possible, HTML tags, scripts, punctuation etc.
- Count the number of words in each of the following dictionaries:
a) altmed terms: such as ‘homeopathic’, ‘herbal’, ‘naturopath’
b) pseudoscientific: clearly suspect terms that scientists rarely use such as ‘toxins’, ‘superfoods’.
c) domain specific words from biomed, physics or chemistry such as ‘energy’, ‘vibration’, ‘organic’.
d) skeptical words: words that no sincere homeopath would ever use, such as ‘placebo’, ‘flawed’, ‘crank’ or ‘prosecution’.
e) commerce terms that would indicate that something is for sale, such as ‘products’, ‘shipping’, or ‘p&p’.
f) Run a few other checks on pomo terms and religious terms, although not much is done with these.
- Compare the ratio of frequency usage of these various types of terms and compare them to preset thresholds. If a threshold is exceeded then append the test’s associate sentence to the response. The tweaking I have been doing to the site has been adding words to dictionaries and varying the thresholds for matches.
This does not always work, Some quacks are very clever and avoid the obvious quack words. Nonetheless they still have completely hatstand ideas.
So, if anyone else has suggestions, then I would be very greatful. Just need to give up my real job to concentrate on this now.