All language subtitles for Adam Ruins Everything s02e15 Science.eng

?

(KNOCKING)

(DOOR OPENING)

Thanks for letting me use the
lab on the weekend, Dr. Sherman.

Of course, Winnie,
we want all the students

competing in this year's
West Haven Science Competition

to feel supported.

How's your research
project coming along?

Oh, it's very promising.

I'm gunning to win this thing.

Oh, great, well, good luck.

I'd offer to stay and help
but Sunday is my fun day.

So, I'm just gonna leave you
here to use the lab

completely unsupervised.

Thanks.

Let's get to work, Carl.

- (CARL SQUEAKS)
- (WINNIE) If my hypothesis is correct,

my project will blow everyone
else's out of the water.

And then I'll get into the
university of my choice...

and be on the path to becoming
a groundbreaking scientist

just like my heroes.

I'll follow in the
footsteps of Marie Curie

and Mimi Phipps Clark.

And do research to benefit mankind.

Because Carl, in a world
filled with ambiguity,

science is always right.

Actually... Science
gets it wrong all the time!

Mother physics!

Wait, are you Adam from TV?

Oh, you watch the show.

Yeah, when "Myth Busters" isn't on.

Now those guys are smart.

Well, wanna do the honors?

Sure.

Hi, I'm Winnie Jones
and this is Adam Conover,

and I'm on "Adam Ruins Everything."

Uh-oh.

?

Corrected & Synced by Bakugan

Adam, I've seen your show.

It's about proving things wrong.

But you can't do that to science.

Science is how we find out
the truth about the world.

Yes, it is.

But the way we do science
is often flawed.

Well, not here, it isn't.

Carl and I are working on a very
exciting, very solid study

about how Vitamin D affects
human memory retention.

Oh.

(SQUEAKING)

Yes, you are a good
little test subject.

Actually, if you're trying
to learn about humans,

he's a terrible little test subject.

(LOUD SQUEAKING)

Sorry.

What? Everyone knows testing on rodents

is practically the same
as doing actual human trials.

Yes, everyone "knows" that.

Too bad they're wrong.

Cool! I've always
wanted to be a B-A-L-B-C-J

laboratory mouse.

But what are we doing here?

The same thing we do
every night, Winnie.

Ruining a common misconception.

(THUNDER CRASHES)

Rodents make up 95% of all animals

used for biomedical
research in America.

That's 20 to 30 million mice a year.

Whoa, they must be such
effective test subjects.

Actually, they're not.

Out of all clinical trials
based on mouse models,

the results don't apply to
humans over 80% of the time.

Well, time to file these.

(ADAM) And those failed trials

can cause hundreds
of millions of dollars a piece.

And this drives up
the cost of development.

Meaning we get fewer new drugs.

Yeah, we're out of funding.
(GRUMBLING THROUGH PHONE)

Well, then tell the cancer patients

they'll have to wait, Pete!

But how can that be?

I read mice share
97% of DNA with humans.

True, but that 3%
makes a big difference.

Mouse metabolisms are seven
times faster.

We have different
inflammation reactions.

And, you know, people are over
3,000 times bigger.

- (GASPS)
- (SQUEAKING)

(ADAM) And our bodies can have

very different reactions to medicine.

In one extreme example
from 2006, a new cancer drug

was tested in mice and
passed with flying colors.

But when human subjects
were injected with just

1/500th of a mouse dosage,

within hours, all of the volunteers

went into catastrophic,
multi-organ failure

and had to be rushed to the ICU.

Oh.

The truth is, testing on mice

just doesn't tell us as much about
humans as we think it does.

Jeez Louise.

So, why do we even use mice
in the first place?

Because they're cheap and convenient.

Compared to other animal test subjects,

mice are a bargain.

And they're great in a lab
setting because they're easy

to tame and store
and they breed like crazy.

Congratulations!

It's 100!

Rodents are also super easy
to genetically customize.

Want a mouse with no immune system?

Or how about one
that glows in the dark.

Extra tumors strike your fancy?

You can have whatever model
of mouse you want.

Finally, the Animal Welfare Act,
the only federal law

that covers testing on animals
doesn't protect mice at all.

(SCREAMING)

Oh, yeah,

we can never get away
with this on a rabbit.

So, even though we know
testing on mice is a problem,

we keep doing it anyways
'cause it's easier?

(WOMAN) And at this point,

it's almost impossible not to.

Ah, Winnie, this is...

Dr. Azra Raza,

Director of the Myelodysplastic
Syndrome Center at Columbia!

I traded six Albert Einsteins for her.

(DR. RAZA) Thanks, Winnie!

And thanks for keeping me
in mint condition.

Medical science is almost
entirely built

around mouse testing.

Researchers have been using mice

since the 1800s.

And students are taught on them

from day one of school.

And most other testing methods

are prohibitively expensive.

You quite literally cannot
do a study without mice.

At the moment in the United States,

the FDA will almost
never approve a trial in humans

unless researchers have
done it in rodents first.

The real problem is cancer
research has become

so mouse centric, we cannot
replace rodent testing

without overhauling the whole system.

Which is a shame.

Because if we want
to cure cancer in humans,

we should be studying
human cancer cells.

We are wasting valuable
resources and time on mice.

(SQUEAKING)

Hey, come on! This is a family show.

(SIGHS)

Thanks, Doctor Raza.

You're welcome.

And here's another tip.

If you put me in your bike spoke,

it will sound like a motorcycle.

(MIMICKING MOTOR PUTTERING)

(SIGHS)

(BOOKS THUD TO FLOOR)

Now my whole project isn't useable.

Well, you don't have to throw
out all of your work.

Nope! I am gonna start
a new project from scratch.

I mean, might as well get used to it.

When I'm a scientist,

I'll be doing nothing
but science all day.

Not quite.

Want me to tell you about it?

Coou have found
a cool kid to mess with?

(WHISTLING THEME)

You know, Winnie, the reality
working as a scientist

might not be quite what you expect.

Very funny, Adam.

You're a real Humphry Davy.

The chemist who discovered the
effects of laughing gas? Hello?

I know exactly what life
as a scientist will be like.

I'll spend my days investigating
the big unanswered questions.

I'll follow the research
wherever it leads,

beholden to nothing and no one

but the pursuit
of pure scientific truth.

That's a beautiful dream.

But in reality, what science
you get to do depends a lot

on who gives you money to do it.

'Cause if you wanna pursue
scientific truth,

first you gotta find someone
willing to pay for it.

Around two-thirds of basic
research is privately funded.

But here's the problem.

The priorities of the people
handing out the funding

often affect what research gets done.

(ANNOUNCER) Welcome to "Smart Tank."

Today the Smarts will hear
a proposal from Winnie,

a young researcher competing

in the West Haven Science Competition.

Hello, Smarts.

Uh, today, I'm asking for funding

for my biomedical research.

Here's my proposal.

You'll find it's very exciting
and worthwhile.

First up, a major funder of research,

corporate America.

Winnie, we might be able to help you.

I just have one question.

Is this gonna help us
sell more candy bars?

No... Why?

Well, corporations and trade groups

tend to fund research
that will make them money.

In 2011, the National
Confectioners Association

paid for a study

that implied eating sweets
actually makes kids skinnier.

If you can't provide
that kind of return

on investment, I'm out.

Well, I see plenty of upside.

Your research can help us
develop new drugs.

Really?

Sure.

And if your results make
our drugs look good,

we'll publish them.

Otherwise we'll bury them

so the results will never
be seen. (CHUCKLES)

Yeah, they actually do that.

Well, that's wrong.

I won't be a part of it.

Oh, an idealist. I'm out.

Well, there's two Smarts left.

Okay, another major source
of funding is philanthropy.

Winnie, I apologize for my
robber baron friends here.

You and I, we're in it to help people.

See? This lady gets me.

Sure, and philanthropic donors
fund some worthy research.

But they're also more
likely to give money

to causes that have
affected them personally.

You have to use this to find
a cure for diamond allergies.

(LOUD SNEEZE)

Or maybe I could study diabetes
or sickle cell anemia?

Oh, sorry, sweetie, I'm out.

This is a travesty.

Isn't anyone willing to fund objective,

independent, unglamorous research?

Yes, there is one final funding source.

Public funding.

Also is Mark Cuban here

because I would, like,
die if I met Mark Cuban.

Because government funding
isn't controlled by profits

or the whims of one person,
it supports the unsexy

yet vital basic research
that all science builds upon.

And it's led to advances like...

Heart rate monitors,
the Human Genome Project,

audio recording,
water purification, satellites,

solar panels, super computers,
GPS and the internet.

Just to name a few.

Even Google got its start
with a government grant.

But you won't see that on the
first page of search results.

Wow, sounds like public
funding is awesome.

Finally a funding
source with no downsides.

Well... there is one little wrinkle.

Federal science budgets are
controlled by politicians.

I hereby award you, Winnie,
this government grant for...

(WHISPERING) What's that, Senator?

You'd rather have tax cuts
than scientific knowledge?

And there's a powerful lobby
that won't like the research?

And you don't know enough about science

to care about the consequences?

(SIGHS)

Sorry, I'm out.

No! This work is worth the money.

Ask any scientist, they'll tell you.

Sorry, politicians tell me
what to do, not scientists.

Maybe not the best system.

But science needs the money.

Yes, it does.

Private funding will never be
enough to support research

in the public interest.

The kind of long-term careful
science that doesn't generate

profits or glory,
but improves people's lives

and increases our understanding
of the universe.

We need public funding for science.

But in the past 50 years,

the portion of the discretionary budget

that government spends
on research and development

has been cut nearly in half.

And with less funding,

there's more competition
for fewer resources.

In the past two decades,

30% of National Health Institute
grants got funded.

But today, it's about 17%.

No, no, no!

And from 2010 to 2013,
the U.S. had the biggest

three-year reduction
of scientific research

and development since
the end of the space race.

(SIGHS)

But if it's so hard to get funding,

then how do we do science?

Well, sometimes we just don't.

In a recent study of 11,000 scientists,

half of them said they've
had to stop doing

valuable research partway through

because of economic pressures.

Who knows what great advancements
we lost out on as a result.

As much as we'd love to believe
that the progress of science

is determined solely
by the best and brightest,

decisions about which
science gets funded

have always been controlled
by non-scientists.

It's all about who holds
the purse strings.

That may be true but science
always triumphs in the end.

Look at all these journals.

Great, factual peer reviewed
research is published every day.

(SUCKING TEETH) Ah...

- Ugh.
- What?

Nothing, just that a surprising
number of scientific studies

do not... hold up.

(SCREAMING)

- Oh!
- Excuse me.

I'm using those studies for my project.

Go ahead, but there's a good chance

a lot of those studies are wrong.

Adam, please, when a study
has been published,

that means its data
is robust and accurate.

Winnie, if you'd ever read
"US Weekly", you'd know

that just 'cause something
is published,

that doesn't mean it's true.

What's "US Weekly?"

Wow, you are nerdier
than I am. (CHUCKLES)

Studies can be flawed
for a number of reasons.

?

If the sample size was too small,

the luck of the draw
could've skewed the results.

We should test this
on 10,000 people but...

I only know ten.

That's okay, making friends is hard.

(ADAM) Or maybe there was
a confounding variable.

Like, if the subject was
using another product

that could affect the results

and the researchers
didn't account for it.

Oh, wow, this pill is really

boosting our subject's
resistance to sunburn.

(ADAM) And something, very rarely,

unscrupulous researchers
will fake their results.

What if we just make up numbers

that show that this pill cures cancer?

Hello? Nobel Prize committee?

I've got some data you've gotta see.

Sure, so we just need a way to find out

which studies are accurate.

We already have one.
And that's to reproduce them.

To show the results
weren't just some fluke,

someone else has to do another study

that tests the same question

to see if they can get
the same findings.

It's like double checking the answers.

Curse, my results were way different.

And here's the problem.

Since studies are usually reproduced

after they're published,

that means they have a chance

to influence other research
or even become famous

before we find out they're wrong.

For instance, remember that
famous study on power posing.

Uh, it's only one of the most
watched TED Talks of all time.

It proved that by standing
in the power pose,

your body produces hormones
that give you confidence.

Well, no one's ever been able
to reproduce those results.

What?

Or how about that series of studies

from the '70s and '80s

that showed you can actually
make yourself feel happier

just by smiling?

Well, that makes sense, right?

Sure, except that in 2016,
almost 20 labs tried

to replicate it and not one of them

was able to reproduce those findings.

(TRAIN WHISTLE BLOWS)

Holy Hawking!

If that's true, then how many
other studies might be wrong?

Oh, a lot of them.

In fact, many scientists
say that we're in the midst

of a reproducibility crisis.

Like a good type of crisis, right?

One research team just
replicated 100 famous psychology

studies but found they couldn't
reproduce about 60% of them.

(BEEP)

Another revisited 67 major drug studies

and found that about 75%
didn't match their results.

And another team zeroed
on 53 recent cancer studies

and couldn't reproduce 47 of them.

(BEEP, LOUD EXPLOSION)

But those numbers are so high!

If science gets it wrong that often,

then we should be reproducing
every study.

(MAN) That'd be nice, but
it probably won't happen.

There... and that's how
you fold a fitted sheet.

This is Professor Nosek.
of the University of Virginia.

And the co-founder
of the Center for Open Science.

Adam's right, Winnie.

Replication studies
are not getting done.

And it's because scientists don't
have enough good reasons to do them.

Wait, what do you mean?

To succeed in their careers,
scientists have to publish

and get attention for their research.

And that's more likely to happen

if their studies have new
and flashy conclusions.

My study found that Fidget
Spinners give you six-pack abs.

(CROWD GASPS AND APPLAUDS)

And the pressure to get
these kinds of results

sometimes leads researchers
to cut corners.

I didn't need test subjects.

I just asked my bro, Kevin.

He's super ripped.

On the other hand, replication
studies make for good science.

But they don't get as much attention.

People just aren't
interested in studies

that don't find anything new.

And I replicated a 2003 study

on classification of
fine-grained polypeptide folds.

The results held up.

(CRICKETS CHIRPING, MAN COUGHS)

Hey, her work is just as important.

Yeah, but it isn't as exciting.

Science is competitive.

There aren't enough jobs or
grant dollars for everyone.

So, if you were trying to get
people excited about your research, what would you do?

Follow-up on someone's prior findings?

Or try to find something new
and amazing?

I, uh, also did a study that found...

candy cures leukemia!

(CROWD GASPS AND APPLAUDS)

(SHUTTERS CLICKING)

These incentives to publish lead
to a lot of unreliable research.

One estimate suggest that in the U.S.,

we spend $28 billion a year
on biomedical research

that can't be reproduced.

That's such a waste!

Well, flashy findings can be useful

for stimulating new areas of discovery.

But we need to change the
incentives so that researchers

are rewarded for producing results

that are interesting and reproducible.

Thanks, Professor Nosek.

No problem.

Now if you'll excuse me,

I'm gonna go use the emergency shower.

I save a ton on my water bill.

Oh, thrifty. (CHUCKLING)

So, even if my research
gets positive results,

I still have to find
someone to reproduce it,

but no one will because
they only care about

their own careers?

Wait, what are you doing?

You're right, Adam.

Science sucks, I'm done.

I'm gonna go drink some
Red Bull in the woods

or do whatever cool kids do.

Oh, wait!

I was about to tell you why
science is actually amazing!

I thought the structure of my show!

?

(BIRDS CHIRPING)

You forgot this.

I don't need it.

I'm quitting the competition.

You know, I used to love
that no matter how chaotic

and uncertain the world felt,

I could at least count on
science to always be right.

But you showed me, it's not.

Winnie, do you know
why science is so awesome?

It's the field of human inquiry

that embraces mistakes
and learns from them.

Scientists don't just assert
what they think is true.

They test their hypotheses
against the empirical evidence.

And if it turns out they're wrong,

they admit it and try again.

The fact that science
seeks out its own errors

and corrects them
is its greatest strength.

But how do we fix what's
wrong with science itself?

With more science. With more science.

When I learned about
the reproducibility challenges

in science, I thought
we need to study this problem.

So we started
The Center for Open Science

to investigate why is it that
researchers are not producing

reproducible science
even though they want to?

So, we built software

so that researchers
can share their data more easily

and be more transparent
in their methods.

We also work with journals
and societies and funders

to align the incentives so that
it's in researchers interests

to be more open
and reproducible in their work.

And when I first emigrated
here from Pakistan,

I had no idea how much research
in the U.S.

rested on flawed mouse testing.

To combat this problem, I started
to collect human tissue,

and this is how I found

a novel treatment for pre-leukemia.

- Whoa.
- So, if something in science is broken

like mouse models or faulty studies,

it doesn't mean we failed.

It means we need people like you
to find the solution by using...

Science!

(ALL) Yes!

You're right!

I have to use science to fix science.

Eureka!

Thank you, guys.

(FAINT CHATTER)

And finally... Winnie.

And your project is...

Wait, where is your project?

Dr. Sherman, I didn't do a project.

I did 19.

Because for my experiment,

I replicated everyone else's research,

and I found that all
of their conclusions were...

irreproducible.

Wow, well, that doesn't quite
follow the project guidelines.

But, nice work, Winnie.

Sorry, everyone else.

(MUFFLED GRUMBLING)

You suck.

Well, I didn't do it for the glory.

I did it for science.

Come on, Adam, I'll show you
my new research proposal.

I got some sweet new binder tabs.

Ooh, I love office supplies.

(CHUCKLING)

(BOY GASPS)

The card moved.

Did anyone else see that?

Holy... (SCHOOL BELL RINGS)

?

Hello, we're here
on set with Brian Nosek

because I have way more
questions for him

about the reproducibility
crisis in science.

Thanks for having me.

Is the crisis real and is it something

that we should all be worried about?

Well, it's at least
a crisis of confidence.

And it's, uh, something that
emerged when researchers

across a variety
of disciplines realized

that the published literature,
what we thing we're doing

and producing and finding isn't
as reliable as we think it is.

And so, can you walk us
through a little bit

of how the fields
realized this was happening.

Well, the oddity is is that
we've known the challenges

for reproducibility for 60 years.

And it hasn't gotten a groundswell.

But that has shifted
over the last ten years

because perhaps the internet
helped make it easier

to evaluate lots of different
papers at the same time.

Perhaps because of a couple
of very dramatic findings

that there has been a convergence.

And so it has become a thing.

Now that we know about it,
we can do something about it.

That's exactly right.

Science is at its best

when it is constantly self critiquing

of looking at where it's limited,

where it's not finding things
that are reliable

and looking for ways to do it better.

Every scientific claim is wrong
in some fundamental way.

And the process of discovery
in science is figuring out

where our current models
of the world are wrong

and how we can make them less wrong.

I'm so fascinated by that.
Can you elaborate on

every scientific theory about the world

is wrong in some way. In some way.

If we had a perfect model of the world,

it would be as complex as the world.

How it is we describe
how the world works,

how people interact,
how atoms interact,

whatever it is, is a simplification.

- It is a way to try to describe...
- Right.

...lots of different things that
are happening in simple terms

that make predictions about
next things that will happen.

Right.

But our predictions
are always imperfect.

They can't quite get exactly
the phenomena as we study them.

So, they're wrong. Right.

But wrong doesn't mean,
oh, that's useless.

It's not information that we need.

It's not information that we can use.

Newton's Laws of Mechanics

are the most successful theory
ever, right?

We use them to build bridges
and skyscrapers...

I learned about them
in high school, yeah.

And they're wrong, right?

Einstein showed how they are wrong

when things are moving very fast.

So, it isn't that something
being wrong means,

oh, can ignore science.

It means there's always
something more to learn.

So, that leads me really nicely
to my next question which is...

Is this a reason that people

should stop believing
or trusting science?

You know, should the audience go home

and every time they
read about a new study say,

oh, that's probably
never gonna be reproduced

and so I'm not even gonna
think about it again?

One of the big challenges
of scientific knowledge

is that is accumulates very slowly.

So, no one study is definitive.

Drinking coffee is bad for you.

No, drinking coffee is good for you.

No, drinking coffee's bad for you.

Well, make up your mind, scientists.

It's those claims that are
initially very exciting.

But then survive confrontation
over time

are the ones that survive
and we have confidence in.

So, as a reader of the newspaper
of reporting scientific results,

one thing to look for is,
is this the first finding?

Is this something brand-new?

Uh, or is it the accumulation

of a lot of different
pieces of evidence?

That's a very easy signal for
whether that's a credible claim

or a reliable claim
that we know a lot about it

versus something new
and possibly exciting.

Yeah.

The reason to trust science

is because it doesn't trust itself.

Ahh. Right?

It's that science is always
questioning its own findings

which makes it the most reliable
way to discover things.

Thank you so much for being here.

Thanks for having me, Adam.

Repair and Synchronization by
Easy Subtitles Synchronizer 1.0.0.0