Evidence is not optional in medicine

July 2013 SAMJ (Source: http://www.samj.org.za)

On Thursday I attended a lecture given by my dad to a cardiac support group on the topic of evidence-based medicine (EBM). It was really good, even though I don’t think he’s given a lecture for over 20 years. While he was preparing I gave him a few resources to help out, such as Trick or Treatment: Alternative Medicine on Trial and one or two papers and blogs that I stumbled upon that looked relevant. One of the papers I sent him was Evidence-based medicine – are we boiling the frog? which is in the latest issue of the South African Medical Journal. It’s only two pages long and open-access so it’s worth reading. It’s one of those interesting times when you agree with everything the person says but then differ on the conclusion. In this case the author’s conclusion is that EBM is important but not necessarily a reflection of the truth.

The author points out a number of known problems with medical research and EBM. These range from fairly minor problems, like the difference between statistical and biological significance, to major issues like data fabrication. It’s important to be aware of these issues or it would be easy to misinterpret the available data. What I did find a bit concerning was the lack of treatment of any of the attempts to solve the problems. For example, publication bias (which ties in with sponsorship by for-profit organisations) was mentioned as a problem for medical literature as only the positive results get reported but there was no mention of All Trials, an initiative to improve publications of clinical trials which seems to having a bit of a positive impact.

Staying on publication bias, it’s at least something that we can detect through funnel plots. If we do this for a particular topic then we can tell whether that is something to worry about. This, of course, requires checking all the available studies, something a doctor obviously can’t do themselves for every medical question but something which should be done systematically. This both can check for publication bias and, since we’ve just collected all the studies, can see whether they agree. This is because more important than the individual studies is what the body of work says when taken together, see the second perspective here. Meta-analyses of many studies on the same topic ought to be more accurate than any single study. In addition it should reduce the effect of the various biases and problems that can affect individual studies. Unfortunately, there’s no mention of using multiple studies in the SAMJ article.

I would’ve appreciated if the article had included a bit on those sort of methods to overcome the problems that will be encountered with EBM but at the very least it’s good that they make people aware of them. Like I said at the beginning though, I don’t agree with the conclusions which say some things that are slightly concerning.

Evidence-based medicine has been defined as ‘The conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.’ There are two major assumptions in this statement. First, it is assumed that the evidence is in fact the best. Unfortunately this is not necessarily so, and published evidence is affected by bias, sponsorship, and blind faith in mathematical probability which may not be clinically relevant.

Here my issue is that they’ve twisted the definition a bit. There is a huge difference between “current best evidence” and “best possible evidence.” The current best evidence may not be the best possible evidence but that’s not a reason to throw it away. We can’t know for certain whether we have the best possible evidence but it’s reckless and irresponsible to just ignore the evidence that we do have. Occasionally that will lead us to make the wrong call but at least we can justify why we made the wrong call.

Second, the evidence is population based and may not be applicable to the individual, and blind adherence to this concept may cause harm. We must not abandon clinical experience and judgement in favour of a series of inanimate data points. Medicine is an uncertain science.

And…

The concept [evidence-based medicine] is disease and not patient orientated, is not scientifically perfect, and must not be viewed as exclusive. As Osler observes, ‘Variability is the law of life and no two individuals react alike and behave alike under the abnormal conditions which we know as disease. The good physician treats the disease, the great one treats the patient.’

The first is from the abstract and the second is from the body but I think they need to go together. These sections worry me because they seem to imply that, because of the limitations and caveats with EBM, that we should return to when everything was based on personal judgement. I don’t think the author is advocating abandoning evidence-based medicine but there is a strange sort of conflict between these statements and supporting it.

Clinical experience and judgement is necessary but to call EBM “a series of inanimate data points” that are “disease and not patient orientated” is very misleading. Patients are individuals who need personalised treatment but we learn how those treatments work by looking at large groups. That’s still completely patient-orientated and necessary to know what will and won’t work. I actually find those comments disturbingly similar to those made when Pierre Louis published his results showing that blood-letting was harmful. As described in Trick or Treatment:

[W]hen Pierre Louis published the results of his trials in 1828, many doctors dismissed his negative conclusion about bloodletting precisely because it was based on the data gathered by analysing large numbers of patients. They slated his so-called “numerical method” because they were more interested in treating the individual patient lying in front of them than in what might happen to a large sample of patients.

Hopefully, we’ve moved on in the last 180 years. The only way to make sure our treatments are anchored in reality is by using the current best available evidence, tempered by knowledge of its limits. This is how we know what works and how we can best treat patients.