so I'm a doctor, but I kind of slipped sideways into research.
And now I'm an epidemiologist, and nobody really knows what epidemiology is.
Epidemiology is the science of how we know in the real world if something is good for you or bad for you.
And it's best understood through example, as the science off those crazy, wacky newspaper headlines on these are just some of the examples.
These are from the Daily Mail.
Every country, the world has a newspaper like this.
It has this kind of bizarre, ongoing philosophical project of dividing all the inanimate objects in the world really into the ones that either cause or prevent cancer.
So here is some of things they said cause cancer recently divorce, WiFi, toiletries and coffee here.
Some of things, they say, prevents cancer crusts, red pepper, licorice and coffee.
So already you can see there are contradictions here.
Coffee both causes and prevents cancer.
And as you start to read on, you can see that maybe there's some kind of political valence behind some of this.
So for women, housework prevents breast cancer.
But for men, shopping could make you impotent, so we know that we need to start on picking the science behind this and what they hope to show is that unpick ing dodgy claims I'm picking the evidence behind dodgy claims.
Isn't ah, kind of nasty carping activity?
It's socially useful, But it's also kind of extremely valuable explanatory tool because real science is all about critically appraising the evidence for somebody else's position.
That's what happens in academic journals.
That's what happens at academic conferences.
The Q and A session after a post op presents data is often a blood bath, and nobody minds that we actively welcome it.
It's like a kind of consenting intellectual S and M activity.
So what I'm gonna show you is all of the main themes, all of the main features of my discipline, evidence based medicine.
On that, I will talk you through all of these and demonstrate how they work exclusively using examples of people getting stuff wrong.
So we'll start with the absolute weakest form of evidence known to man, and that is authority in science.
We don't care how many letters you have after your name in science.
We want to know what your reasons are for believing something.
How do you know that something is good for us or bad for us.
But we're also unimpressed by authority because it's so easy to contrive.
This is somebody called Dr Gillian MCI's Ph.
D or to give her full medical title.
Gillian MMA Keith Again.
Every country has somebody like this.
She is our TV diet guru.
She has massive five series of prime time television, giving out very lavish and exotic health advice, she turns out, has a non accredited correspondence course, PhD from somewhere in America.
She also boasts that she's a certified professional member of the American Association of Nutritional Consultants, which sounds very glamorous and exciting.
You get a certificate and everything.
This one belongs to my dead cat, Hetty.
She was a horrible cat.
He just goes the website, fill out the form, give them $60 arrives in the post now.
That's not the only reason that we think this person is an idiot.
She also goes on on says things like You should eat lots of dark green leaves because they contain lots of chlorophyll, and that really oxygenate your blood on anybody who's done School.
Biology remembers that chlorophyll and chloroplasts only makes oxygen in sunlight on.
It's quite dark in your bowels after you've eaten spinach.
Next, we need proper science.
Proper evidence so red wine could help prevent breast cancer, is the headline from The Daily Telegraph in the U.
A glass of red wine a day could help prevent breast cancer.
So you gonna find this paper and what you find is it is a real piece of science.
It's a description of the changes in the behaviour of one enzyme when you drip a chemical extracted from some red grape skin onto some cancer cells in a dish on a bench in a laboratory somewhere.
And that's a really useful thing to describe in a scientific paper.
But on the question of your own personal risk of getting breast cancer, if you drink red wine, it tells you absolutely bugger.
All okay, actually, turns out that your risk of breast cancer actually increases slightly with every amount of alcohol that you drink.
So what we want is studies in riel human people on.
Here's another example.
This is from Britain's leading diet, a nutritionist in the Daily Mirror, which is our second biggest selling newspaper.
An Australian study in 2001 found that olive oil in combination with fruits, vegetables and pulses offers measurable protection against skin wrinkling.
So then they give the advice.
If you eat olive oil and vegetables, you'll have fewer skin wrinkles, and they very helpfully tell you how to find the paper.
So you gotta find the paper on what you find is an observational study.
Obviously, nobody has ever been able to go back to, like, 1930 get all of the people born in one maternity unit and half of them eat lots of fruit and veg and olive oil, and then half of them eat McDonald.
And then we see how many wrinkles you've got.
Later, you have to take a snapshot of how people are now, and what you find is, of course, people eat fruit and veg and olive oil have fewer skin wrinkles.
But that's because people who eat fruit, virgin olive oil, they're freaks.
Okay, they're not normal.
They're like you.
They come to events like this, right?
They are posh, they're wealthy.
They're less likely to have outdoor jobs there.
Let's like to do manual labor.
They have better social support.
They're less like to smoke.
So for a whole host of fascinating, interlocking social, political and cultural reasons, they are less like to have skin wrinkles.
That doesn't mean but it's the vegetables or the olive oil.
So ideally, what you want to do is a trial, and everybody thinks they're very familiar with the idea of a trial.
Trials are very old.
The first time was in the Bible, Daniel, 1 12 It's very straightforward.
You take a bunch of people, you split them in half.
You treat one group one way you treat the other group the other way, and then a little while later, you fold them up and see what happened to each of them.
So I'm gonna tell you about about one trial, which is probably the most well reported trial in the U.
K news media over the past decade.
And this is trial official pills, and the claim was official pills improve school performance and behavior in mainstream Children.
And they said, we've done a trial.
All the previous trials were positive, and we know this one's gonna be, too.
That should always ring alarm bells, right, because if you already know the answer to your trial you shouldn't be doing one.
Either you've rigged it by design or you've got enough data.
So there's no need to randomize people anymore.
So this is what they were gonna do in their trial.
They were taking 3000 Children.
They're gonna give them all these huge fish oil pills, six of them a day.
And then a year later, they were going to measure their school exam performance and compare their exam performance against what they predicted.
Their exam performance would have bean if they hadn't had the pills.
Now, can anybody spot floor in this design on DNO?
Professors of clinical trial methodology are allowed to wants to this question, so there's no control.
OK, there's no control group, but that sounds really techie, right?
That sounds really no.
That's a technical term.
The kids got the pills and then their performance improved.
What else could it possibly be if it wasn't the pills, they got older.
Okay, we will develop over time.
And of course, also there's the placebo effect.
Receiver effect is one of the fascinating things in the whole of medicine.
It's not just about taking a pill and your performance and your pain getting better.
It's about our beliefs and expectations.
It's about the cultural meaning of a treatment, and this has been demonstrated in a whole raft of fascinating studies comparing one kind of placebo against another.
So we know, for example, that two sugar pills a day are a more effective treatment for getting rid of gastric ulcers.
Van One sugar pill a day two sugar pills a day beats one sugar pill a day, and that's an outrageous and ridiculous finding.
But it's true.
We know from three different studies on three different types of pain.
That a saltwater injection is a more effective treatment for pain than taking a sugar pill taking a dummy pill that has no medicine in it, not because the injection or the pills do anything physically to the body, but because an injection feels like a much more dramatic intervention.
So we know that our beliefs and expectations can be manipulated, which is why we do trials where we control against a placebo, where 1/2 of the people get the real treatment and the other half get placebo.
But that's not enough.
What I've just shown you are examples of the very simple and straightforward ways that journalists and food supplement pill peddlers and naturopaths can distort evidence for their own purposes.
What I find really fascinating is that the pharmaceutical industry use exactly the same kinds of tricks and devices, but slightly more sophisticated versions off them in order to distort the evidence that they give to doctors and patients in which we use to make vitally important decisions.
So, firstly, trials against placebo Everybody thinks they know that a trial should be a comparison of your new drug against placebo.
But actually in a lot of situations, that's wrong, because often we already have a very good treatment that is currently available.
So we don't want to know that your alternative new treatment is better than nothing.
We want to know that it's better than the best currently available treatment that we have, and yet repeatedly you consistently see people doing trials still against placebo, and you can get license to bring your drug to market, with only data showing that it's better than nothing, which is useless for a doctor like me trying to make a decision.
But that's not the only way that you can rig your data.
You can also rig your data by making the thing that you compare your new drug against really rubbish.
You can give the competing drug in too low a dose so that people aren't properly treated.
You can give the competing drug into higher dose so that people get side effects.
And this is exactly what happened with antipsychotic medication for schizophrenia.
20 years ago, a new generation of antipsychotic drugs were brought in, and the promise wants that they would have fewer side effects.
So people set about doing trials of these new drugs against the old drugs.
But they gave the old drugs in ridiculously high doses 20 milligrams a day of haloperidol.
And it's a foregone conclusion if you give a drug at that high a dose, that it will have more side effects and that your new drug will look better.
10 years ago, history repeated itself interestingly, when risperidone, which was the first of the new generation antipsychotic drugs, came off copyright so anybody could make copies.
Everybody wanted to show that their drug was better than risperidone.
So you see a bunch of trials comparing new antipsychotic drugs against risperidone at eight milligrams a day again, not an insane dose.
Not an illegal dose, but very much at the high end of normal until your bounds to make your new drug look better.
And so it's no surprise that overall industry funded trials are four times more likely to give a positive result than independently sponsored trials bumped.
And it's a big.
But it turns out, when you look at the methods used by industry funded trials that they're actually better than independently sponsored trials, and yet they always manage to get the result that they want.
So how does this work?
How can we explain this strange phenomenon?
Well, it turns out that what happens is the negative data goes missing in action.
It's withheld from doctors and patients, and this is the most important aspect of the whole story.
It's at the top of the pyramid of evidence.
We need to have all of the data on a particular treatment to know whether or not it really is effective.
There are two different ways that you can spot whether some data has gone missing in action.
You can use statistics or you can use stories.
I personally prefer statistics.
So that's what I do.
This is something called a funnel plot on a phone Parts a very clever way of spotting.
If small negative trials have disappeared have gone missing in action.
So this is a graph of all of the trials that have been done on a particular treatment on As you go up towards the top of the graph.
What you see is each dot is a trial on as you got to talk.
Those are the bigger trials.
So they've got less error in them, so they're less likely to be randomly false.
Positives randomly negative.
So they all cluster together.
The big trials are closer to the true answer.
Then, as you go further down at the bottom, what you can see is over on this side, spurious false negatives.
And over on this side, the spurious false positives.
If there is publication bias If small negative trials have gone missing in action, you can see it on one of these groups.
So you can see here that the small negative trials that should be on the bottom left have disappeared.
This is a graph demonstrating the presence of publication bias in studies off publication, bias and I think that's the funniest epidemiology joke that you will ever hear.
That's how you can prove it statistically.
But what about stories?
Well, they're heinous.
They really are.
This is a drug called Reebok sitting, and this is a drug which I myself have prescribed to patients.
And I'm a very nerdy doctor.
I hope I go out of my way to try and read and understand all the literature I read the trials on this They were all positive.
They were all well conducted.
I found no floor.
Unfortunately, it turned out that many of these Trans were withheld.
In fact, 76% of all of the trials that were done on this drug were withheld from doctors and patients.
Now, if you think about it, if I tossed a coin 100 times and I'm allowed to withhold from you the answers half the times, then I can convince you that I have a coin with two heads.
Okay, If we remove half of the data, we can never know what the true effect size of these medicines is.
And this is not an isolated story.
Around half of all of the trial data on antidepressants has been withheld, but it goes way beyond that.
The Nordic Cochrane group were trying to get hold of the data on that to bring it all together.
The Cochrane groups are an international nonprofit collaboration that produce systematic reviews of all of the data that has ever been shown, and they need to have access all of the trial data.
But the companies withheld that data from them, and so did the European Medicines Agency for three years.
This is a problem that is currently lacking a solution and to show how big it goes.
This is a drug called Tamiflu, which governments around the world have spent billions and billions of dollars on, and they spend that money on the promise that this is a drug which will reduce the rate of complications with flu.
We already have the data showing that it reduces the duration of your flu by 30 hours, but I don't really care about that.
Governments don't care about that.
I'm very sorry if you have the flu.
I know it's horrible, but we're not going to spend billions of dollars trying to reduce the duration of your flu symptoms by half a day we prescribe these drugs, we stockpile them for emergencies on the understanding they will reduce the number of complications, which means pneumonia and which means death.
The infectious Diseases Cochrane Group, which are based in Italy, have been trying to get the full data in a usable form out of the drug companies so that they can make a full decision about whether this drug is effective or not and they've not been able to get that information.
This is undoubtedly the single biggest ethical problem facing medicine today.
We cannot make decisions in the absence of all of the information, so it's a little bit difficult from there to spin in some kind of positive conclusion.
But I would say this only think that some light is the best disinfectant.
All of these things are happening in plain sight and they're all protected by kind of force, field of tediousness.