When I decided to stop writing something every day, after a year of writing this blog, I thought that I might start making more use of Twitter by drawing attention to items of interest that didn’t merit a 1200 word consideration.
A story in The Herald, Scotland - ‘Revealed: Heroin causes more harm to Scotland than any other drug’1 - seemed, at first glance, to fit that category but as I read into it I realised that it required more comment than 140 characters would allow. I also realised that I should go to the primary source to better understand what it was about.2
The story, reporting research looking into the comparative harms of psychoactive substances in Scotland, according to the Herald ‘aimed to reproduce previous work led by former chief drugs adviser to the UK Government, Professor David Nutt’.
The paper itself says that;
‘The aim of this study was to obtain a comprehensive consensus from addiction experts in Scotland on the relative harms of drug misuse, both legal and illegal using the ranking system developed by Nutt et al.’
The slight difference in wording is quite significant. ‘Reproduce previous work’ suggests that all aspects will be replicated whereas ‘using the ranking system’ limits the overlap to one aspect. Why the work by Nutt et al3 was not completely replicated is not made clear. This would have given a direct comparison between the two studies.
More importantly, Nutt et al produced a further study of relative harms4 based on 16 criteria not the 9 originally used that Taylor adopted for his work. This is significant, because Nutt undertook the second analysis to deal with criticism of the first paper based on its giving equal weight to personal harm and societal harm.
There is a problem comparing Nutt 2007 with Taylor because Taylor has some important differences in the substances chosen, for example including Ritalin but excluding Catha edulis, khat. The methodology of the Taylor study is also very different from that used by Nutt 2007 and 2010 and raises concerns about its applicability to the totality of Scottish society.
Nutt 2007 used two panels, one of psychiatrists with the second, larger panel made up of people with a broader spread of substance use experience. The second group held meetings to determine a consensus view of harm rankings using a delphic analysis. The results of this second group showed a good conformity with those of the first.
Rather than use this approach, Taylor et al opted for surveying a much larger number of people involved in dealing with addiction using either an online questionnaire or a face to face interview putting the same questions. Though described as ‘clinical experts’, the detailed breakdown of the 292 participants shows 133 ‘Addiction community psychiatric nurses’, 39 ‘Addiction workers’ and 52 ‘Social workers’.
In other words, over three-quarters of the responses were from people at the very frontline of dealing with problem substance use. The danger this creates is that the study is ranking harm at the second level. That is, rather than ranking the overall harm caused by a substance you end up ranking the relative harm amongst those who have been harmed enough to seek help from addiction treatment facilities.
For Nutt 2010, the analysis was conducted by the Independent Scientific Committee on Drugs (ISCD) with the help of two experts and took the form of a full day’s meeting. The important difference in the later analysis is that different risks were weighted. For example, Nutt 2007 viewed the social harm of intoxication as being equal to the physical harm of chronic use.
The results it produced were somewhat different from Nutt 2007 and very different from what Taylor has reported. Significantly, it ranked alcohol as the most harmful substance overall where Taylor ranks it only fourth. That is a counter-intuitive conclusion given that alcohol misuse in Scotland is widely acknowledged as being more of a concern than it is for the UK as a whole.
Taylor argues that there are problems with the results of Nutt 2010. One of these, the paper says, is that ‘substances such as alcohol, nicotine and volatile agents are far more widely available’ ignoring all the evidence from street prices of drugs that shows availability is rarely an issue.
Perhaps because of these objections, Taylor ignores the Nutt 2010 results when saying that cannabis was rated 11th by Nutt and his colleagues whereas the Scottish study found it to be the least harmful of all. Taylor suggests that this difference could be because the participants in the current study do not deal with cannabis users as often. That argument looks a little flawed when you see that tobacco was rated as 7th most harmful substance in spite of it not being the sort of substance addiction treatment facilities deal with.
That dilemma for the authors of how to explain the placing of Cannabis sativa, marijuana, brings out a disconnect in the reasoning. The Herald report has Dr Taylor making a point of the higher number of participants over Nutt 2007 and I took from that an inference that the higher number made the study more valid. When explaining the result for cannabis, however, Taylor seems to be suggesting that the ‘experts’ who participated weren’t best placed to reach valid conclusions.
I want to mention one final point that may, initially, appear trivial. The introduction to the paper says ‘Class C includes cannabis’. Now, I am most certainly not mocking the authors for making an elementary error. We all make mistakes and I’m quite sure this website has an abundance so I would be foolish to do so. But there is an important difference between this site and a paper in BMJ Open and that is peer review. It is really quite surprising that the reviewers did not detect this error.
One of the first tests for deciding what is science and what nonscience in published work is whether a paper has been peer reviewed. If peer review is not rigorous, then that distinction becomes less valid.
I have a second concern. The error occurs in the introduction to the paper. I have written repeatedly about the tendency of authors writing about ricin, the toxic extract of Ricinus communis, castor oil plant, to include claims, in their introductions, about the history of ricin that do not stand up to even a few moments scrutiny. I’m concerned that introductions are not taken as seriously as they should be and this error could be an example of that.
Taylor et al set out to produce a harm scale for Scotland to match that in Nutt 2010 and they use that harm scale to say that the existing drug classification system is unscientific, something that is already widely accepted. Unfortunately, the methodology used results in a flawed harm scale and I am concerned that basing a valid conclusion about the classification system on that flawed harm scale provides ammunition for those who see no reason to question existing drug policy.
Revealed: Heroin causes more harm to Scotland than any other
drug The Herald 5th August 2012
2. Quantifying the RR of harm to self and others from substance misuse: results from a survey of clinical experts across Scotland BMJ Open 2012;2:e000774 doi:10.1136/bmjopen-2011-000774 24th July 2012
3. Development of a rational scale to assess the harm of drugs of potential misuse The Lancet, Volume 369, Issue 9566, Pages 1047 - 1053, 24 March 2007
4. Drug harms in the UK: a multicriteria decision analysis The Lancet, Volume 376, Issue 9752, Pages 1558 - 1565, 6 November 2010
You can send comments via the contact page but please be sure to say what blog entry you are commenting on.