#1273 Helen Pearson - Beyond Belief: How Evidence Shows What Really Works

00:00 Ricardo Lopes: Hello everyone. Welcome to a new episode of The Dissenter. I'm your host, as always, Ricardo Lops, and today I'm joined by Helen Pearson. She's a science journalist, author, and chief magazine editor for for the journal Nature, where she oversees the journal. Lisbon opinion content and today we're going to talk about her latest book Beyond Belief How Evidence shows what really Works. So Helen, welcome to the show. It's a big pleasure to everyone.

00:28 Helen Pearson: Thank you so much for asking me to be on the show. I'm happy to be here.

00:32 Ricardo Lopes: OK, so as the title or uh the subtitle of your book implies, we're going to talk a lot about evidence today and why have evidence is so important. But what is actually the premise of your book? I mean, how would you characterize it?

00:50 Helen Pearson: I would say it says that in a world full of misinformation and conflicting claims, then evidence from science can show us what's nonsense and what's not, I guess, um. But the, the, the, the, the easiest way to actually describe it is to probably tell you how I came to write the book. Can I, can I just do that? OK, so, so, so, so the story is, um, so during my previous book, which was, um, called The Life Project, I, I met and interviewed this remarkable person who, um, called Ian Chalmers, who is a doctor and researcher in the UK. Um, AND he kind of told me his story, and this kind of introduced me to the, the history of the modern evidence movement. Um, SO, so his story is that he was training to be a doctor in the UK in the 60s and early 70s, um, and he was in obstetrics and gynecology. And he noticed something odd about the profession, which was when two doctors gave out advice for the same condition, they would often give out very different advice. So, you know, one doctor might say bed rest is really important, or prenatal vitamins, or, you know, not eating or drinking during labor, and another doctor would say no such thing. Um, AND, and what would often happen in medicine generally was that everyone would just follow the advice of the most senior doctor in the room, and, um, that's sometimes now called eminence-based medicine. Um, SO, uh, to, to Chalmers, this just seemed really puzzling, like, you know, which treatment was most effective and which doctor was he supposed to believe. And then he came across this influential book in 19 published in 1972 by a doctor called Archie Cochrane. Um, AND Cochrane argued that the best way to find out whether one treatment is more effective than another is to use rigorous, um, evidence from experiments or from observational studies, and he also argued that the most, um, sort of unbiased form of evidence is from randomized controlled trials where, um, scientists assign people, uh, the same types of people to. To um control and treatment groups and then see whether the treatment group improves faster. So, so this was really inspiring to Ian Chalmers. He'd never even heard of randomized controls, uh, before, um, even though he trained to be a doctor, right? Um, AND, and because of this, he, he decided to do this really pioneering work where, um, he went off and brought together all of the controlled clinical trials that had been done that tested. Um, TREATMENTS in pregnancy and childbirth, and, and not only did he bring them together, but he synthesized them, so he, he brought together similar studies, um, uh, to try and sort of see what the the entire body of knowledge showed. Um, AND this basically revealed that many practices in pregnancy and childbirth were either not based on evidence or actively doing harm, um, so for example, um, Bed rest wasn't based on anything particular, you know, any particular strong evidence base, um, even invasive processes like shaving women's pubic hair during labor or conducting episiotomies, which is a surgical incision. Um, WERE, were not based on solid evidence, and so, um, you know, so countless women had been subjected to these highly invasive and degrading practices, but just because doctors assumed they were the right thing to do. And, and, um, and, and Ian Chalmers' work helped bring an end to them, and it became part of this bigger movement towards evidence-based medicine. It was, it was really influential in that, um, of, of moving medicine from this this eminence-based approach into this evidence-based approach. So, sorry for the long story, but it's sort of, through, through someone's, you know, story, we can kind of see how that, that whole field made the change. And by the way, and that, that's what inspired the book. To go back to your original question.

04:29 Ricardo Lopes: No, no, yes, of course, I mean, then we're going to go back to the history of evidence-based medicine, particularly later on, but Archie Cochrane is the one that who gave the name to Cochrane Libraries, right?

04:45 Helen Pearson: Exactly, exactly, yes, yeah,

04:47 Ricardo Lopes: yeah, yeah, yeah, yeah, OK, so I mean, but, but why should people care about evidence and particularly nowadays, as you said at the very beginning, since the internet is filled with misinformation, medical misinformation, scientific misinformation more generally, and Very unfortunately there are more and more people apparently moving away from evidence-based science. I mean, what would you tell those people? I mean, how, why should they care about evidence?

05:23 Helen Pearson: Well, I think they should care now more than ever because, exactly because of the problem you just talked about, um, you know, when the world is filled with all this misinformation on, on social media. Evidence provides us with a way to, you know, to, to, to guide us towards what, what really works. Um, SO it, it, it's, it's all it's empowering. I mean, I really wanted the book to be empowering, to, to help people understand, you know, why the world had moved towards evidence in the first place and perhaps how they could use it in their own lives.

05:54 Ricardo Lopes: OK, and uh let's uh let me ask you a very basic question, but which I think is very important for us to explore here. What is evidence? I mean, what do we mean by evidence?

06:08 Helen Pearson: Yeah, it's such a great question, and I think there's lots of kind of um confusion about it and, and uh people use it in different ways. So, so the dictionary definition of evidence is um information which indicates whether a belief or proposition is true or valid. Um, SO it's, so it's information, right, and, and I think that, that probably when a lot of people talk about it, or certainly, you know, when, when the scientists I interviewed talk about it, they, they use it as being synonymous with evidence from science, so from rigorous kind of studies. Um, BUT there are other forms of evidence, so there's evidence in a court of law, which is just information which is produced in that court. Um, THERE'S evidence. From lived experience, right? People's experiences in the world is also information. And, and so, um, so when I was, I mean, I did have to define it at the beginning of my book because I am mostly talking about scientific evidence, um, and, you know, often that is considered the most rigorous kind of, uh, form of evidence, but I think it's really important actually to not invalidate other forms of knowledge, and maybe we'll come onto that. Um, BUT, but the key, I think that the way that the, the term is sort of, I think, can lead to confusion is, um, that because it has this sort of wide range of meeting, you know, meanings to just mean like, oh, a bit of information, a bit of data, then sometimes people attach it to things, um, you know, to say something is evidence-based, when maybe it, it, it's not as rigorous as, as we might hope. So you sort of see the term evidence-based pop up all over the place. Um, LIKE, I, I searched on Amazon the other day, um, and found, uh, Evidence-based book, this is books, evidence-based horsemanship, um, evidence-based golf, um, and so, you know, maybe there's some great science in those fields. I'm not, I'm not saying that those books are bad. I'm just saying, you sort of see it attached a lot to things, and, and it doesn't necessarily mean what the kind of real advocates of the evidence movement are, are talking about, I think.

08:02 Ricardo Lopes: But how is evidence established? I mean, how do we establish that a particular piece of information is evidence or should be treated as such?

08:13 Helen Pearson: Well, in science, it's because the evidence would have been produced by particular methods which scientists hope produced something which is trustworthy. So, um, I, I already mentioned randomized controlled trials, which is sometimes held up as a kind of, you know, gold standard method. Um, BECAUSE it tries to, but by the way it's done, it tries to even out all the other factors that could, let's say, um, cause a, cause an illness, um, or, or lead to a, um, a, a change, um, so that we can have real confidence, right, that, that the result is, um, is caused by, let's say, a drug. Um, SO, so it's through these methods, I think, which scientists have developed over years, which, um, and, and through the kind of rigor with which they've analyzed that data, that, that we have more confidence in those pieces of information. But, I mean, even within science, you know, there are different sort of, I mean, some people, you know, put them in hierarchies, there are different kind of tiers, I suppose, of, of, um, rigor or reliability that scientists attach to different forms of evidence.

09:12 Ricardo Lopes: And historically, when exactly did people start caring more about evidence when it comes to how medicine and science are approached, and I mean, how did things work before that?

09:28 Helen Pearson: Well, in a way, I think people have kind of always used evidence of a type, right, because I mean, throughout human history, we've wanted to know whether something that we do on, on a personal level, like people will have been observing, did this work, like, you know, did it work when I, I don't know. ATE these berries, or did they make me sick, I mean we're, we're accruing that personal experience all the time, but in terms of sort of doing more systematic studies, I mean that was sort of really, sort of example, the first randomized control trials, I mean very early ones were, were being done by, um, Doctors really, uh, sort of in the 1700s, um, and so there was this famous, for example, early randomized trial done by James Lind, I don't know if you might have heard this one, he became very famous for, um, doing this, this very basic, well, arguably not randomized, but a controlled trial, um, for looking for treatments for scurvy. He was on the HMS Salisbury and treated 10 sailors with, with different types of um of, of treatments, including oranges and lemons, and um discovered through that that it looked like, um, having citrus fruit helped combat the, the symptoms of scurvy. And, and so from that point, you know, scientists in a way were just becoming. Over time, more interested in just collecting data, I think, on, on large numbers of patients, and then over time, these methods evolved, um, but, but really, I mean, you know, randomized controlled trials have only, um sort of really taken off since about the, the 60s, I would say, 60s to 70s. It's quite a recent phenomenon.

10:57 Ricardo Lopes: Yes, but, but it's not like, as you alluded to there, that people in societies prior to that, even in more traditional societies, don't care about or don't apply methods that allow for them to gather evidence about whatever kind of topic they're interested. And even for their practical lives, right? I mean, there's not a strict division between, oh, OK, so at a certain point science developed and we started caring about evidence and prior to that no one cared about evidence at all. I mean there's not that strict division or

11:41 Helen Pearson: well, no, I mean it goes, it goes back to what you were saying of what you count as evidence. If you count experiential evidence of things that we're doing every day, and observing, well, I did this, you know, I ate this, and I felt good, um, you know, then, then that, that counts. We, we've been doing that for, for sort of, you know, millennia, um, but at some point, you know, that there's a switch to going, OK, we can get more rigorous evidence if we move beyond these experiential or anecdotal observations and collect information on large numbers of people. I mean, that's what our scientists are generally trying to do. Um, BECAUSE of course the more data and the more people you observe, then the more confidence you can have in that what you're observing is a kind of general principle. But yes, I mean, I think a gradual evolution is, is probably correct.

12:21 Ricardo Lopes: So earlier you talked about Ian Chalmers and Archie Cochrane. Tell us a little bit more, tell us a little bit more about the history of evidence-based medicine. I mean, how did it develop and how did it arrive at the stage we are in today?

12:41 Helen Pearson: Sure, so, so I, yeah, I, I told you the story of, um, Ian's story, which was, you know, what I encountered when I was, I was writing the book, and, um, I mean, I'll go on and finish that story because that was sort of one big important part, I think, of moving, um, medicine from being this eminence-based, you know, most senior doctor in the room approach to the evidence-based one. And so after he'd done this, um, synthesis of all of. The evidence on pregnancy and childbirth, um, he was inspired to go on and, and do that same practice, um, across all medical specialities. Um, SO, so, you know, what's the evidence on effective treatments in heart disease and cancer and diabetes and so on. And, and this was the kind of, um, concept behind a group called the Cochrane Collaboration, um, which Ian, And others founded in 1993, um, and that group has become very well known and, and still goes on today for producing these kind of, um, gold standard, what are called systematic reviews, which is a, which is a very rigorous, um, analysis and combination of evidence from randomized trials, um, and, and systematic reviews from, from Cochrane. And also from many other scientists are now, um, formed this kind of bedrock, almost like this invisible bedrock of evidence, um, and, and on the basis of that, doctors create clinical guidelines, and then the clinical guidelines are what's used generally, um, by doctors to, to help make decisions in the clinic. Um, SO, so that was kind of one really important part. But then there was this other story taking place at the same time in, in North America, which. Another, um, key character called David Sackett, who was also training in medicine in the 60s and also became really frustrated with the eminence-based, um, medicine approach. Um, AND he moved to McMaster Medical School in Canada, and, um, started teaching medicine in this different way. So rather than just, um, telling students to do what the most senior doctor said, um, he, he taught them to, to solve problems using research. Let's go to the library, uh. And, and see what's been published on this topic. And that evolved, um, into this approach, which actually got the, which they called evidence-based medicine. Um, AND, and that term evidence-based medicine, uh, was only published in the medical literature in 1991. So, again, you know, quite recent. And from that point, there was something about giving it this name, and at that moment in time, it kind of raced around the world. And, um, you know, within a decade or so, it, it was being called one of modern medicine's greatest intellects. ACHIEVEMENT and it's become really the, the standard way in which modern medicine is practiced. So most people who've been to the doctor, you know, will have been treated according to these principles of, um, of evidence-based medicine, um, even though they might not be aware of it. And, um, and that was kind of, that was kind of one really big motivation for writing the book, because I felt like it had touched like most people's lives, and yet, you know, they're, they're not really outside academia, it's, it's not really well known.

15:35 Ricardo Lopes: So nowadays in terms of experimental designs, randomized control trials, I, I think are the gold standard in medicine and science more generally. But what were the origins of randomized control trials? I mean, how were they developed?

15:55 Helen Pearson: So, so, so I talked um about one of the early ones with um with James Lind, and, and over time, you know, those methods evolved, but I, I would say another, another landmark randomized control trial which people, you know, talk about in, in the history of medicine was. This key one that was done um in the 40s, um to test streptomycin, so it was kind of the early days of antibiotics. Um, AND, uh, streptomycin, I think was only the second antibiotic which was was uh being considered after penicillin. Um, AND this trial was done, this, this was sort of really pioneering in randomization terms, because one of the problems with, or the difficulties with randomization is that even, is that how do you randomize people? So, so, you, you want to randomize people, so that you're, you're, um, assigning equal numbers with, you know, um, the same type of condition, or, or, or the same, you know, age to these different groups, right? So, everything about these groups is hopefully the same, apart from the fact one's got a treatment and one doesn't. Um, BUT the problem is, it's actually quite difficult to do. So, even if, like, doctors are like, OK, you know, I'll, I'll, the first person goes in the, in the control group, and the next person goes in the treatment group, um, sometimes then biases, human biases creep in, and they sort of, um, uh, you know, consciously or unconsciously start. Putting, let's say, the people who are a bit more sick in the treatment group, because they kind of want them to get better. Um, AND, uh, Austin Bradford Hill, who was this very famous, uh, well, became a famous statistician, uh, realized this was a problem, and, and he realized that blinding was important in randomization. So in this streptomycin trial, what they did was, um, got this series of, of envelopes, um. And, uh, as each person was enrolled in the trial, so these envelopes are unmarked, they would just have to pick up the next envelope, open it up, and, and it would say, um, S for streptomycin or C for control. So even the doctors who were assigning the patients to the trial couldn't know in advance what, what group they would go into. And over, so, so that was considered a real landmark of showing how. Important it was, um, to, you know, to, to make sure that there was what's known as, as fully random allocation in, in these trials. So, so, so that was like one, you know, really important landmark, um, but just, just to go back to something I said about, you know, why they took off in medicine, I mean, that was due to another kind of, um, uh, basically a, a, a tragedy in, in science around the thalidomide drug, um, so, uh, thalidomide was given to women with, with morning sickness, um. And uh in the US, uh the, the FDA didn't, didn't initially approve thalidomide because the, the um person who, who was um. RESPONSIBLE for approving it said actually, um, you know, I, I'd like to see a bit more safety data, basically, um, and because of that, it, it became, later it became apparent, um, that thalidomide was unfortunately, um, responsible for causing, um, disability and, and many, many, uh, deaths in, in newborn babies, um, and so, um, you know, it helped kind of avert that tragedy in the US but also helped the FDA. Bring in rules which said, OK, from now on in order to approve drugs, we're going to need to see much more convincing data around the effectiveness of these drugs. Um AND over time, like by the end of the 60s, it had become standard for that evidence to be demanded in the form of randomized trials. And so from that point, you know, once regulators started demanding that this was supplied, um, then the use of randomized trials took off and, and that kind of, you know, Um, sort of, I guess, cemented it, shall we say, as, as the gold standard in medicine.

19:26 Ricardo Lopes: How about meta-analysis, because of course today it, it sounds very obvious to scientists and science communicators that we should do meta-analysis that instead of just looking at one single study or a couple of studies, we should Look across uh the best quality studies out there and try to figure out what the bulk of the evidence tells us instead of again just relying on one single, even if it is high quality study. How did people come up with that idea?

20:04 Helen Pearson: So, again, quite, quite a recent, um, methodology, actually. And, and, I mean, you say it sounds really obvious, but I don't know that, that, I don't know that it, that it, it does in terms of, like, you know, I mean, a lot of, I mean, I'm a science journalist. A lot of science journalists, I think, are just chasing it still after the, the latest study, right? I mean, that's why you get this phenomenon of, you know, red wine's good for you one week, and then it's bad for you the next, because it's just like, well, this is What one study said, and I think a lot of scientists too, I mean, they're incentivized to publish the next paper, not to go back and do a careful systematic review and meta-analysis. So, in my view, I mean, I, I, I've become this huge fan of systematic reviews and meta-analyses through writing this book, but, um, you know, I don't think they get the glory, right, that, that randomized trials do somehow, even though they're, as you said, they're so vitally important, um. But the, the, the meta-analysis, um, I, I believe it was announced in a hotel ballroom about 1974, um, and it was devised by a, um, Uh, a, a researcher called Jean Glass, who, uh, became interested in whether psychoanalysis was really, um, or psychotherapy was really effective for treating mental health disorders, because he had, he had actually had this therapy himself and was very convinced that it was effective even though other reviews, which weren't so rigorous were suggesting that, that it wasn't effective. Anyway, he, he went off and did this like massive search. Through the, the scientific literature, and you have to remember, you know, in, in those days, and really until quite recently, it, uh, it, it's very, it was very, very difficult and laborious, right, to collect together all of the studies which have been done on a topic, because you have to go to the library and literally find these things by hand in, in paper journals. Um, AND, and he, and he want, and so he developed this method by which you, um, can, Convert all of the different studies, even though they're slightly different and might have studied different outcomes into this single, um, metric called effect size, um, and, uh, you know, developed these statistical techniques, um, which became known as meta-analysis in order to, to understand to combine these studies and see what they say as a whole. Um, SO I sort of describe it as being a way to extract, um, the signal from the noise.

22:11 Ricardo Lopes: Uh, SO, uh, I mean, uh, randomized controlled trials, meta-analysis, systematic reviews, it seems to me that these are all ways of trying to minimize as best we can, uh, human, uh, error and human bias, particularly, right? I mean, because, of course, as humans, people tend to have their own, uh, pet theories. That hypothesis and uh I mean unless people try to do a randomized control trial or something in terms of design similar to that uh I mean people could very easily try to manipulate the results to make their hypothesis seem more plausible or with better or better supported so um I mean it's just ways of trying to prevent that. Right.

23:06 Helen Pearson: Yes, yes, it is really, but, but it's absolutely not foolproof. I mean there are lots and lots of, of, of bad randomized trials in the world, uh, probably more than are good ones, and so, and that's what's interesting, so, uh, you know, with a systematic review, what you, let's say you want to know whether, I don't know, you know, a statin is effective, you go off and find all of the drug, the, The randomized trials which have been done on statins. I mean, you might come back with like tens of thousands or more, um, of studies, and then you have to filter them down for to, to the ones which are actually good enough to include. And then they have a whole step where you look for risk of bias, you know, that, that remains in the study, because, because scientists are human, right? So, of course, these biases creep in. Um, SO a systematic review in a way is, is a, is a, an attempt to whittle down the randomized controlled trials to the ones that we care about, but then of course there's lots of really bad systematic reviews, um, which are done too, um, which are biased or the methods aren't that good, so, so that, I don't know, I mean, just as a journalist, you know, I, I, I kind of wrestle with that now actually, because I, I have become much more, I mean, I, I try and, you know, take my own medicine and, and, and use more systematic reviews now in my writing to look at the, the body of evidence as a whole, but it's sometimes it's very difficult for me as a, as a non-expert to know whether this is a really good systematic review or there's some hidden conflict of interest or bias that I can't see, which of course is where it's still important to be interviewing people.

24:32 Ricardo Lopes: So in the book you go through several examples of domains of people's practical practical lives where evidence can also apply or be used in different ways, and one example that you talk about is social policy. I mean, in what ways can evidence perhaps improve social policy?

24:55 Helen Pearson: Yeah, so I guess in parallel to, to this um embrace of evidence in medicine, there has been, um, well, sometimes in parallel and sometimes after, there's, there's been this movement in all these other fields that I looked at to also um use evidence to figure out what really works, so what practices work, and in the book I look, as you said, at policy and. Policing and conservation and all these other fields, they are often really like inspired by evidence-based medicine, right, they, they sort of look in their own field and say, hm, you know, our, our practices are all based on anecdote and conventional opinion too, shouldn't we be using evidence to find out what works? I mean, in, so in social policy, the idea is that you, um, you know, you can test whether a policy works just the way that you would test whether a drug works, so can we do a randomized experiment and give some people the, the, the, the, you know, the new program and some people not, and um, I mean, 11 early famous example was this um test of a, a program called Progressor in Mexico. This has become a kind of famous um randomized trial in, in social policy, um. And this was in the 90s and it was a new initiative to see if, you know, families in poverty could be helped. Um AND some families were randomized to get this, this program um in which they would receive payments. The family received payments, but in order to receive them, it came with conditions, so families had to send their children to school, um, the, you know, mothers had to go to health clinics. And, and, and, you know, children had to receive vaccines and so on. So it's, it's called a conditional cash transfer now. Um, AND sure enough, this, this, you know, very innovative and pioneering uh randomized trials suggested that these families were helped long term compared to the control group. And that, that whole idea of conditional cash transfers, because, you know, partly because of this convincing evidence has helped that be embraced across different countries. Um, AND that, and that's gone on. I mean, there's just now there's been this sort of huge movement in, um, particularly in development economics to, to test, uh, policies, you know, um, does it work to give incentives to help families have, um, to, to help, you know, encourage families to have vaccines, for example, a whole range of things which, which are now tested in randomized trials to see whether they're effective.

27:09 Ricardo Lopes: Um, YOU mentioned policing. Uh, WHAT, uh, in what ways can evidence be applied there also, because nowadays people worry, uh, a lot about and with good reason, I think, um, about biases, uh, uh, in policing, um, like for example, uh, racial biases, gender biases, and things like that. So how can evidence be used to improve policy?

27:40 Helen Pearson: I mean, that, that was a whole field that I found so fascinating to look at, because I, I mean, I just, I, I hadn't reported on policing before, but there's this huge global movement in evidence-based policing, um, which again follows the same approach. There was the same realization by some pioneers in the field that, you know, a lot of practices in policing are based on anecdote and conventional wisdom or doing what we've always done, you know, could we, could, could they use the ideas from medicine to test what works, um. And, and, and so a lot of that has been done. I mean there's thousands now of studies which test practices, you know, everything from like, you know, do body worn cameras work, um, to hotspots policing. I mean that's like a success story of evidence-based policing, um, so the, the sort of standard practice, um, a few decades ago was to do, um, random patrolling, so, you know, police drive around the streets or roam around the streets, kind of looking for crime. In the idea that that's going to, to reduce it, um, but actually through a series of, of much more rigorous tests, um, it's been shown that actually focusing police on the, sometimes, you know, on the hotspots, as in the places where crime is, is particularly high, even, you know, that might just be a street corner, for example, um, is much more effective at reducing crime overall. Um, SO it is starting to change police practices, um, quite a lot actually, in some parts of the world.

28:59 Ricardo Lopes: Uh, Education is also a field that unfortunately seems to be very permeable to pseudoscience. For example, I've already had conversations on the show about that with people who study that phenomenon of pseudoscience in education, and I mean, people are very unfortunately open to ideas like learning styles and Um, I, I mean, different ways that, uh, people can be intelligent. I mean, the, the, um, the different types of intelligences out there and stuff like that. So, I mean, what would you say to people that work in education? As to why it's very important for them to really evaluate evidence properly in terms of what works and what doesn't when it comes to educating children and adolescents.

30:00 Helen Pearson: Well, I, I think the reason it's important is it's, it can help children more. Um, I mean, again, there, there's a vast number of really rigorous studies in education, evidence-based education as it's, as it's sometimes called, or evidence informed, um, that, you know, show that, that children um move forward faster um using certain practices and, and, um. I, I agree with you. I mean, I think it's interesting because there have been certain fields I've looked at where it just seems like, you know, sort of fashions and fads are really commonplace, um, and education is probably one. I mean, parenting is another, um, so, and, and also business and management. I mean, you know, sort of where books come out, it's like, well, you know, where's the evidence behind them? But it, but I mean, in parenting, I've become a really big fan of a, of a, British organization called the Education Endowment Foundation, which has become quite a world leader in, um, synthesizing, just like Cochrane does for medicine, but synthesizing the evidence in education, and they have a brilliant tool, um, called the Teaching and Learning toolkit, which makes that very accessible to parents and teachers. And so, and, and so if you look at that, you can actually rank different approaches. That's what they do. To say, OK, you know, if I, if I invest this certain amount of money as a school, uh, which, which technique will, will move my, my students forward, uh, by the most, on average? Um, AND the top of the list is, is an approach called meta-cognition, um, and meta-cognition, um, is basically when children think about, learn about how they learn, or think about how they think. So, for example, Um, if a child, um, solves a math maths problem, and then they go, hmm, OK, the way it really helped me to write out my working. Like, they, they were thinking about how they learn, and so they're, they're, you know, able to use that in the future. And, and the evidence suggests, I think it would, that moves children forward if it's done properly, like by, you know, 7 to 8 months in an academic year, more than they would otherwise move ahead. Um, AND also feed feedback is also really an effective approach. So, um, you know, so, I mean, that's why, right, uh, that's what teachers are trying to do. Um, SO it seems, it seems a shame if, if that evidence is not being used. I'm, I'm not saying, by the way, I just, just to caveat that, I, I think it's really important to not, I don't know, point a finger of blame, do you know what I mean, at practitioners? I mean, teachers are really, really busy people who are absolutely doing their best they can for children, so I don't think it's a fault necessarily of teachers. I mean, it's more a sort of fault of, of the system maybe, that in medicine, doctors learn about evidence when they train, and they're provided with these, you know, evidence-based guidelines. It's already sort of there for them, it's embedded in their practice, and, and, and education and some of these other fields, you know, haven't, haven't got that far yet. Um, SO, you know, maybe that's where we, we might see some progress in the future.

32:49 Ricardo Lopes: When it comes to parenting, which you also mentioned there, I think it's one of those areas where people are, I mean, not everyone, of course, but many people are not really that willing to listen to people who want to or they think or they interpret as wanting to telling them how they should parent their kids. I mean, people are really Resistant to that in the sense that they think that they are the ones who know better in terms of the, for example, the knowledge and the values they should instill in their children. So why do you think that people should perhaps be more open to listening to the ideas of people who look at the evidence when it comes to parenting?

33:37 Helen Pearson: Well, well, I, I mean, I, I absolutely respect that, you know, view, and I, and I don't, I don't think any evidence advocate's going to get very far if they try and, you know, force facts down the throats of, of resistant people, right, that's, that's not really the, the way to go. You have to listen compassionately to people's concerns and perhaps present, you know, the evidence in the hope that it might be something they want to use. I mean, So, so parenting, I think is quite interesting, and I was interested in it, of course, myself as a parent, um, having, as, as so many of us have, right, just encountered this sort of huge overwhelm of conflicting information and books and all this stuff that you, that you get as a parent and wondering what I was supposed to be doing, um. So I, I think there's a huge appetite for, for parenting skills and advice which, which are, um, based on evidence, and to be, and, and, and so there are, as, as in all these other fields, there are some really kind of pioneering people who've come up with, for example, um, evidence-based parenting interventions, which is basically just a way of, you know, like, Train, you know, a, a sort of training course for parents which, which says, well, actually, um, there's, there's quite a lot of evidence behind these particular approaches and they can empower you as a parent and also hopefully, I mean there is evidence suggests that they lead to, to, you know, better behavior and um in, in, uh, in children. Um, AND some of it is very, some of it aligns anyway, it's not like, I, I think what scientists are suggesting, it's not like it's radically different from what most parents are doing. A lot of it is very commonsensical, you know, responding compassionately to children, um, creating boundaries so that children understand, you know, what's right and wrong, very clear communication about what, what parents want from children, so it's nothing really radical. I just think it helps you kind of, you know, sift out all the noise of, of advice, that's all.

35:26 Ricardo Lopes: OK, so, uh, before moving on to other topics, uh, let me just ask you briefly about climate change. I mean, in terms of, um, I mean, evaluating evidence in the ways that we should approach how we tackle climate change, what would you say are the best approaches out there?

35:47 Helen Pearson: Well, that's, yes, that's a great question, because, um, I'm not, so, so that's really where some sort of, um, It's a bit of a frontier in evidence synthesis, I would say, and I wrote about that a bit in the book. So what is clear from synthesizing evidence is, is, you know, that, that humans are, are, are causing climate change. Uh, WE, we, we record this as we're sitting in a massive heat wave, um, and, um, and, and, and that's been done with this, you know, huge effort by the, the IPCC, um, but there's an interesting initiative now to say, OK. Well, you know, we know that, but what's less clear, um, is what are the most effective ways to tackle climate change, and there's a, there's a great initiative now taking off, which wants to do, take the kind of Cochrane approach of synthesizing evidence on what works in medicine, and put it into climate solutions, and saying, there's lots of research out there on what could work in climate policy, but what can we synthesize the evidence and say, what, what, Would work the best in these, in particular contexts, um, and that's a group called the What Works, um, Climate Solutions, which, which is still getting it's, um, still sort of forming and it looks like it's gonna really feed into the next IPCC report. So I think that's a really exciting, um, development, of course, we need governments to listen to those solutions, um, but at least the, the research will hopefully be there.

37:05 Ricardo Lopes: So earlier we talked about randomized controlled trials, systematic reviews, meta-analysis. So there's a hierarchy of evidence, right? I mean, what is this hierarchy of evidence and why does it matter? I mean, why should there be a hierarchy when it when it comes to evaluating evidence?

37:27 Helen Pearson: Yeah, there, there are lots of hierarchies of medicine, and that emerged in the kind of early days of evidence-based medicine, um, I think when doctors were trying to sort of sort out what's the most trustworthy, um, and generally how it works. I mean, it's, it's, it's done in different ways and pitched in different ways, but generally there's a pyramid, um, and you've got normally kind of systematic reviews and meta-analyses of randomized trials at the top, because we talked about that, like gold standard evidence synthesis, then you've. Randomized trials, um, then you might have, um, very large observational studies, um, then you might have ones with smaller numbers, like case control studies, um, you know, individual case studies. Um, AND often at the bottom of these original ones would be, like, doctor's experience. But I, I think, I mean, I think, you know, people have criticized that. So, so, and that's because of some of the things we talked about. I mean, just because it's, it's a randomized trial. Doesn't mean it's a, it's a good study, it can be a bad randomized trial, and also I think it sort of really downgraded the importance of some forms of evidence, like, like doctor's experience, which is actually a really, really important part of the decision making process. So I, I, I feel like there's a much more kind of inclusive approach now actually to, to evidence, um, and it's more about how we bring together these, these different forms and understanding that different forms of evidence are going to suit different, different. Um, SITUATIONS, you know, sometimes you, you can't do the randomized trial, you just, you simply can't, it's impossible, it's too expensive, it would take 20 years to produce results, um, and so we need to make, make, you know, do or, or, or bring together the best evidence that, that we've got. Um, BUT in terms of how they got their, their positions in the first place, um, you know, again, it was sort of, um, I mean, partly just because people become really enamored with the randomized trial, but it, but, but if it's done really well, it can eliminate these other, uh, you know, factors or confounding factors and really help us understand causes, basically.

39:18 Ricardo Lopes: Uh, OF course, at the very bottom are expert opinions, and nowadays even people, uh, put on the picture an even lower layer where they talk about anecdotal evidence from non-experts. I mean, how do you think that people should deal with expert opinion? Because very unfortunately nowadays in the in this kind of more anti-intellectual environment that we live in, people tend to, or certain people tend to very easily dismiss experts just because they're experts. I mean, how do you think that people should deal with them?

40:00 Helen Pearson: Well, I mean, in the evidence world, um, you know, in, in, in all the fields I've looked at, I mean, expert opinion or, or experience is, is, is, is absolutely part of the, the decision making process. So, so in medicine. The, the sort of, the principles of evidence-based medicine are, you know, you, you are, you take evidence, and you've got the experience of, of the clinician, really important, right, that they've learned all this, everything that they've learned to people, and of course, you've got the, the, the values and preferences of the patient themselves, and it's those three things together which would lead to a decision. So, so you're already using these other forms of evidence, and that's the same in all of these other fields too. I mean, look at policy, right? Um, I mean, if, so, so let's say, uh, research shows that, um, this particular climate policy works, um, you know, that doesn't mean it was necessarily the one that's going to be introduced, because politicians who are making that decision or policy makers have all these other things to consider. They have to consider politics. And, and public opinion, you know, so, so, so evidence is one part of the pie in the decision making process. Um, AND, and so even, so I guess I'm saying that because even within, you know, I, I think it's sometimes a misconception actually around evidence that it, that just provides simple answers, but it doesn't. Um, I mean, the, the, the rejection of, of experts, which is sort of slightly different, I mean, that's this sort of populist idea, isn't it, that, um, That which has, which has drawn in scientists, right, which is sort of um the idea that that scientists are part of this elite of experts and they don't, um, understand or or appreciate or out of tune with, um, with popular opinion or the kind of experiences of, of normal people. And I mean, I think that's, you know, that's been a sort of politically fueled, er, position, um, which, which is absolutely important to understand because it's sort of leading to some of this kind of distrust and rejection of evidence today.

41:54 Ricardo Lopes: Right, so you've already ended up answering my question about whether whether evidence should be the only factor in decision making, but what do you make of forms of knowledge that are non-scientific or at least non-scientific. According to the standards of, uh, Western science, I think we, we could call it. I mean, what do you make of, for example, uh, in the what we nowadays call indi indigenous knowledge and other forms of knowledge acquisition.

42:27 Helen Pearson: Well, I, I think that's a, it's, it's a really important question, and I think it's one that the kind of evidence movement that I write about is wrestling with, is, is how, so, so it, it's, it's very important to, to include, I think, um, so let's say for example, you want to know whether a, um, Uh, a, a nature, I mean, I talked about this example in the book, but, but, you know, what, what's the impact of putting a nature reserve in a particular place when it might, um, involve the displacement, let's say, of a, um, of an indigenous group, Well, you have to understand the lived experience of people there, of the value to them, of that environment, um, of everything they understand about that environment and, and that ecosystem. So, so I think it has to be part of the, the, the, um, The consideration, it, it, it's, it's another form of evidence, but the challenge for, for scientists who are maybe coming from a sort of a, a, a, a, a different approach is how do you integrate these completely different forms of knowledge, and I, and I think that's just this really fascinating question which, um, you know, which, which people are still wrestling with, researchers are still wrestling with, um, so, so yes, I mean, I, very important and perhaps unanswered, I suppose, is, is my answer to your question, we're not quite sure how to approach it yet.

43:39 Ricardo Lopes: Yeah, but I mean, uh, you would agree that at least a priori we shouldn't just, uh, dismiss those forms of evidence and knowledge, right? Because, uh, I, I, it seems to me that very unfortunately, um, there are some scientists out there who tend to have this very this very. I would call it elitist approach to science and knowledge where even for example when anthropologists, cultural anthropologists, for example, talk about indigenous knowledge and perhaps how we should try to integrate some of it with the standard science, they Immediately dismiss that as pseudoscience or anti-science or non-science, something like that. So I mean, I don't think that that kind of approach and attitude is very productive or, or is

44:36 Helen Pearson: it? Yeah, I, I completely agree with you. Yeah, and I mean, I don't know, maybe I speak to more enlightened people, but um. Respect for different forms of knowledge and, and the same way that there's more conversation between different fields in science, um, and an understanding that you, you need all these different forms of knowledge if we're going to solve some of these difficult problems.

44:59 Ricardo Lopes: Mhm. So would you say that there's been a crisis in people believing in evidence? Would you label it as a crisis?

45:11 Helen Pearson: Oh, I love that you've asked me this question because I'm, I'm, well, I'm going back to my desk after this to, to send to press a story I'm publishing next week about whether there's a crisis of trust in science, so, um, this is top of my mind. Um, SO is there, Is there a public crisis of trust in science? So, so having looked at some of the data, um, I would sort of say no, but maybe yes, um, because if you look at like, um, if you look at big surveys that have been done globally, um, trust in, in science and scientists consistently comes out quite high actually, um, in the, you know, like if you ask people, like if you were sort of ranking professions, like scientists generally come right near the top with like, Teachers and doctors, and that hasn't really changed over the years. Um, HOWEVER, um, there, there are problems, clearly, um, and one of them is that there's a political polarization around trust in science going on in the US. There are hints of it elsewhere, but very strongly in the US where trust in science has declined amongst Republican leaning people, and it's stayed steady amongst Democrats, and, um, you know, people I speak to say that this sort of, um, It is sort of being fueled by, by politicians, but also now this is sort of being used to attack the scientific enterprise in the US, so the sort of idea is like, well, um, you know, we're, we're going to dismantle these institutions of science because they're not trustworthy. Um, SO that's like one big problem, um, and then the other is there is some, some research suggesting that people are more likely now to question or reject, um, evidence on like contentious issues like vaccines, so, so that does seem to be going on. Um, AND part of that, from what I, you know, understand, and of course observe anecdotally, is that scientific information is just being drowned out really online, right, because before, there were just less sources of information and it was sort of, people would turn towards trustworthy sources, because there weren't very many of them. And now it's becoming much harder to like filter what, what to believe, um, certainly on social media. So, so some people I speak to say, well, scientists are losing influence, um, they're not necessarily losing trust, it's just their voices aren't necessarily, or, or the scientific information isn't necessarily being heard.

47:20 Ricardo Lopes: So what do you think can be done about that? In what ways can we improve people's trust in scientific experts and, uh, I mean, and prevent and perhaps try to prevent as better, as best as we can the dissemination of uh pseudoscience and anti-science.

47:44 Helen Pearson: I mean, some of that is really, is really complex, I think, and, and I feel like there's just so much conversation today about, you know, how to combat misinformation, you know, how to address this point, point you're raising, and some of it comes down to difficult things about technology platforms, right, because on, on social media platforms, the algorithms tend to promote posts which evoke outrage and emotion. Um, uh, YOU know, not necessarily posts which are talking about kind of dry data or scientific facts, so that's one issue, you know, around algorithms, I think, um, you know, and maybe holding technology platforms to account. But, but on an individual level, I, I, I wrote this quite, um, interesting story about how to, um, what to say to a vaccine skeptic. Like, this is on a much more. Kind of personal level, um, because I thought that, like, lots of our readers would, would maybe, you know, like me, have these playground conversations where you talk to someone who's got a very different view to you and might be questioning vaccines, and then I would end up thinking, I just, I don't actually know what to say. But of course, researchers have studied that problem. Um, AND, and really, the answers were, um, you know, don't dismiss them. Um, THEY, you know, people have very valid questions around vaccines that often come from places of concern, of course, um, you know, ask curious questions, you know, why do they feel that way? Um, TRY and offer, if, if you can, um, you know, correct. Information, such as, you know, the, the, the benefits of vaccines outweigh the, the harms for, for most people, according to, like, hundreds of studies, um, and even just offer your, your opinion or, or what you did, um, or the choices that you made. So it's very, it becomes very sort of, um, You know, I think a lot of these approaches about how to deal with, with kind of trust in science and, and misinformation and is, is around, um, of course it's around like human conversations, you know, it's quite relational, um, so, I, I, you know, that that has to be part, part of the answer, I think.

49:39 Ricardo Lopes: OK, great. So the book is again Beyond Belief How Evidence shows what really Works, and of course I will be leaving a link to it in the description of the interview. And, Helen, just before we go apart from the book, would you like to tell the audience where they can find you and your work on the internet?

50:01 Helen Pearson: Uh, YES, you can connect with me on LinkedIn. Uh, YOU can find my stories published, um, at Nature.

50:08 Ricardo Lopes: OK, great. So look, thank you so much for taking the time to come on the show. It's been really fun to talk with you, and I, of course, recommend your book to my audience. It's a fantastic book, and I think that everyone should run and buy it and read it. So thank you so much for doing this again.

50:26 Helen Pearson: Thank you so much. It was really fun talking to you and thanks for your really, really good questions.

50:31 Ricardo Lopes: Hi guys, thank you for watching this interview until the end. If you liked it, please share it, leave a like and hit the subscription button. The show is brought to you by Enlights Learning and Development done differently. Check their website at enlights.com and also please consider supporting the show on Patreon or PayPal. I would also like to give a huge thank you to my main patrons and PayPal supporters, Perergo Larsson, Jerry Mulleran, Frederick Sundo, Bernard Seyaz Olaf, Alex, Adam Cassel, Matthew Whittingbird, Arnaud Wolff, Tim Hollis, Eric Elena, John Connors, Philip Forrest Connolly. Then Dmitri Robert Windegerru Inasi Zu Mark Nevs, Colin Holbrookfield, Governor, Michel Stormir, Samuel Andrea, Francis Forti Agnun, Sverggoo, and Hal Herzognon, Michel Jonathan Labrarith, John Yardston, and Samuel Curric Hines, Mark Smith, John Ware, Tom Hammel, Sardusran, David Sloan Wilson, Yasilla Dezaraujo Romain Roach, Diego Londono Correa. Yannik Punteran Ruzmani, Charlotte Blis Nicole Barbaro, Adam Hunt, Pavlostazevski, Alekbaka, Madison, Gary G. Alman, Semov, Zal Adrian Yei Poltontin, John Barboza, Julian Price, Edward Hall, Edin Bronner, Douglas Fry, Franco Bartolatti, Gabriel Pan Scortez or Suliliski, Scott Zachary Fish, Tim Duffy, Sony Smith, and Wisman. Daniel Friedman, William Buckner, Paul Georg Jarno, Luke Lovai, Georgios Theophanus, Chris Williamson, Peter Wolozin, David Williams, Di Acosta, Anton Ericsson, Charles Murray, Alex Shaw, Marie Martinez, Coralli Chevalier, Bangalore atheists, Larry D. Lee Junior. Old Eringbon. Esterri, Michael Bailey, then Spurber, Robert Grassy, Zigoren, Jeff McMahon, Jake Zul, Barnabas Raddix, Mark Kempel, Thomas Dovner, Luke Neeson, Chris Story, Kimberly Johnson, Benjamin Galbert, Jessica Nowicki, Linda Brendan, Nicholas Carlson, Ismael Bensleyman. George Ekoriati, Valentine Steinmann, Per Crawley, Kate Van Goler, Alexander Ebert, Liam Dunaway, BR, Massoud Ali Mohammadi, Perpendicular, Jannes Hetner, Ursula Guinov, Gregory Hastings, David Pinsov, Sean Nelson, Mike Levin, and Jos Necht. A special thanks to my producers Iar Webb, Jim Frank, Lucas Stink, Tom Vanneden, Bernardine Curtis Dixon, Benedict Mueller, Thomas Trumbull, Catherine and Patrick Tobin, John Carlo Montenegro, Al Nick Cortiz, and Nick Golden, and to my executive producers, Matthew Lavender, Sergio Quadrian, Bogdan Kanis, and Rosie. Thank you for all.