All language subtitles for AI.Confidential.with.Hannah.Fry.S01E03.The.Assassin.and.the.Algorithm.1080p.HEVC.x265-MeGusta[EZTVx.to].en

REPORTER: Good morning, New Yorkers.
Another beautiful day in the city.

The Marriott hotel,
midtown Manhattan.

The CEO of
the largest health insurer in America

sets out for a meeting of investors.

But as he reaches the venue,
a masked man approaches from behind.

Breaking news - we're just hearing

that the CEO of UnitedHealthcare,

which is the largest
healthcare company in the country,

was shot and killed
right here in New York.

Brian Thompson was shot
just before 7am.

Police believe the shooting
was a targeted killing.

The shooting of the father
of two led to a nationwide manhunt.

Five days later,
a suspect was arrested.

26-year-old Luigi Mangione became
a controversial media sensation.

Luigi Mangione, who has also been
dubbed America's Hot Assassin.

Instead of denouncing the killing,
some seemed to cheer it on...

ALL CHANTING: We the people
want Luigi free!

..furious about
the US health insurance industry.

But what few people realise

is that this is a story
about artificial intelligence...

..and the growing use of AI
in modern healthcare.

They're using AI to maximise
the profits off of their clients.

What happens when
life-and-death decisions

are no longer made by doctors,
but by machines?

ADVERT: Artificial intelligence -
a machine beyond the mind of man.

For decades, scientists have dreamed
of creating incredible machines

that could talk like us,
learn like us, think like us.

But what we didn't imagine
is the impact they would have on us.

In this series,
I'm exploring what happens

when AI collides with human lives,

unearthing stories far stranger than
we could ever have imagined.

I'm Professor Hannah Fry,
and for years,

I've been interested in
how AI could transform healthcare.

What I couldn't predict is
that it would take me

to the scene of a murder
on the streets of Manhattan.

Hey, Hannah. Hi.
How are you doing? Good.

Nice to meet you. I'm Hannah.
Lovely to meet you.

Hannah Parry is a journalist for
Newsweek who lives in New York

and was on duty
the day Brian Thompson was shot.

When did you first hear about it?

Well, we first heard the reports of
a shooting up in midtown,

but it quickly came through that
it was something more than that.

We were able to see
that it had been a targeted attack.

Hi, I'm Brian Thompson,
CEO of UnitedHealthcare,

and welcome to the attendees of
Reuters Total Health.

Our mission...

This polished corporate video

shows Brian Thompson
addressing a conference

18 months after he became
CEO of UnitedHealthcare,

America's largest insurer,
with over 50 million customers.

On the morning of his death,
he was on his way

to the company's annual
investor meeting on West 54th Street.

This is the entrance to the hotel
where the conference was being held,

and that's the surveillance camera
that captured the shocking footage.

GUNSHOT

There were multiple bullet casings,

and that kind of became
one of the key parts of the case.

The suspect had inscribed
three words on them,

which were the words
"delay, deny, depose."

Those words are extremely similar to

a very well-known critique of
the insurance industry -

rather than prove claims, they would
rather delay, deny, defend.

It appeared clear that the suspect

had something to say about
the insurance industry.

In the months prior to the shooting,

reports were circulating about
UnitedHealthcare's use of AI

in deciding when to pay out
to patients and when to deny claims.

A new investigation alleges
one healthcare giant

may have been giving the algorithms
too much power.

UnitedHealth pressured
its medical staff

to cut off payments in lockstep with
a computer algorithms calculations.

This news report's from 2023,

which is a year before
Brian Thompson was shot.

I think if you don't live in America,

you've probably never even heard
of UnitedHealthcare

until this Luigi story surfaced.

But there is a lot
going on with them.

And at the heart of it all
are algorithms

and artificial intelligence.

The potential for AI

to transform healthcare
for the better is enormous,

and some of it is already here.

A step toward
medical superintelligence -

that's what the CEO of Microsoft
is calling

the company's
new artificial intelligence tool.

From accelerating drug development...

A revolution in drug discovery

could cure cancer
in the next 50 years

by bringing new approaches
to once impossible problems.

..to reading scans
and detecting diseases...

See that little square? Yeah.

It finds a very subtle polyp
that maybe you would have missed.

..and assisting doctors
during complex operations.

A 3D map of a patient's pelvis
is generated from a CAT scan

to help guide the surgeon
in real time.

In the UK, AI is already being used
in the NHS

to detect the risk of leukaemia

and to identify
early signs of lung cancer.

For our overstretched health service,
this technology has huge potential

to boost efficiency
and bring down costs.

The revolution
in artificial intelligence

offers a golden opportunity to
deliver better care at better value,

and the NHS will usher in
a new age of medicine,

leapfrogging disease so
we are predicting and preventing it,

rather than just diagnosing
and treating.

In the US,
healthcare is a huge business.

I wanted to find out more about
the claims

that UnitedHealthcare
had been using an AI

to make crucial
life and-death decisions...

..so I went to meet a doctor

working in the oldest
homeless shelter in Los Angeles.

OK, so let me take a look.

Mary Marfisee is a medical director
at the Union Rescue Mission.

SPANISH TRANSLATION:

OK?

Mary. Hi. Hi.

Nice to meet you. It's lovely
to meet you. How are you doing?

I want to give you a little tour.

Mary has spent her career

providing medical care
to the city's most vulnerable.

But in 2023,
when her family needed help,

she believes they found themselves
at the mercy of AI.

Mary, is that your hat?
It's my husband's hat.

Oh, that's lovely. I thought
I'd bring it with me today.

Isn't that one of the most handsome
older white guys?

He looks so English. He was Welsh.
HANNAH LAUGHS

He was American-Welsh.

The story with your husband -
when did this all start?

Going back about three years ago,

I noticed he was having
a lot of balance problems,

just falling too many times,
and this was a very athletic man.

And then finally, one fall
that was so bad fractured his nose

and his face was filled with blood

and had some cranial fractures
with it.

Got him in the hospital
and finally got a diagnosis,

which was accumulation of fluid
on the brain.

Oh, my gosh. Right.

Oh, wow. That's a proper fall.
Yeah, it was rough.

So, what happened?

So, we stayed in the hospital

and then he got into
a rehabilitation centre.

And then,
within just a couple of weeks,

I was told by the centre,

"OK, his time is up.
He's no longer in need of services."

And I said, "What?
He can't even brush his teeth.

"He can't go to the bathroom
on his own."

And I thought,
"Something's not right here."

So, we got him home, and then again,

another fall
within just a few months.

And the same thing I was told by
the centre, "OK, his time is up."

And any time
I would complain about it,

I'd hear, "Oh, well, at his age,"

and I got tired of hearing
that preamble.

So, when he was discharged,

if he's not in a fit state to walk
out of the hospital, what happened?

Wheelchair.
Just wheelchair to the car.

Staff takes him out, loads him
in my car and I drive away.

And how'd you get him
in the house on the other end?

It was a struggle.

One time he fell back on me.
I dislocated my right shoulder.

Cos this is just not a man
who's strong enough to be back home.

Right. No, it wasn't good.

And that's when I started calling
UnitedHealthcare, saying,

"Who's making these decisions?"

And the response I would get is,
"Clinical team." Right.

And I started to say things like,
"Well, who runs the clinical team?"

And then I would just get
the run-around.

So I called the ombudsman and that's
when I was told AI is involved.

OK.

After each fall, he wasn't
getting enough physical therapy

or occupational therapy.

You just get weaker and weaker
over time. Yeah. Yeah.

You think he didn't need to die
when he did?

I don't think so.

He had goals to live much longer.

I really thought he could get back
to some of his function.

Had he been given the care he needed?
Yes, I think so.

How angry are you about all of this?

It's more.
I miss him more than anger.

I don't want it to happen
to anybody else.

Thanks for coming.
Thank you so much.

I got to give you a hug because
it meant so much to have you.

Stay in touch.
I'll send you some things.

Thank you, thank you.

There's a kind of irony
to this story.

You have somebody
who spent her entire life

advocating for people's health,

trying to make sure that they get
the care that they need.

When she needed it,
when her family needed it,

despite having paid for it,
it wasn't available.

Frank and Mary's case
was not an isolated one.

By early 2023,
the story was starting to get out -

alleging the widespread use of
AI algorithms by big insurers,

including UnitedHealthcare,
to deny elderly patients care.

Later that year,
there was a Senate inquiry.

The reason we're here today
is that all too often,

the big insurance companies

have been failing seniors
when they need care.

And perhaps most troubling of all,
there is growing evidence

that insurance companies
are relying on algorithms,

rather than doctors
or other clinicians,

to make decisions
to deny patient care.

Two months after the Senate committee
published its report,

Brian Thompson was shot.

Let's head to New York
because police there

are continuing their hunt
for the gunman

who killed the boss of one of
the biggest companies in the world.

The shooter took off
down an alleyway around 55th Street

and is currently still at large.

In the days after the shooting,

the hunt for Brian Thompson's killer
gripped the nation.

Police are looking for a suspect

described as a man about 6'1"
wearing all black.

Police drones, helicopters
and thousands of CCTV cameras

are combing the city
street by street.

On Thursday,
detectives shared new images

of the man they want to question.

The image
caught on surveillance camera

shows him standing at
the check-in desk at a hostel.

The man appears relaxed and smiling.

Just a few minutes later,

he shot Mr Thompson dead
before making his escape.

Then, five days after the killing,
police received a tip-off

from an employee
at a McDonald's in Pennsylvania.

REPORT: The alleged killer has been
identified as Luigi Mangione.

As Luigi was led into court
after his arrest,

it was clear he had something to say.

Luigi Mangione,
the man accused of killing

the US healthcare insurance
chief executive, Brian Thompson,

appeared in New York
to face 11 state criminal counts

that could lead to
a death penalty sentence.

Then something extraordinary
happened.

PROTESTERS CHANTING: Free Luigi!
Free Luigi!

Almost immediately, protesters
flocked to his court appearances.

hailing him as a folk hero.

I'm here because I support
universal healthcare for all,

and I'm here because I believe that

Luigi Mangione's civil rights
are being violated.

So, what brought you here today?

This case is a case about humanity.

I think everybody in the crowd
identifies with

the extortionist nature
of our American healthcare.

They implemented, like, an AI bot
to review the claims.

Some of those people
had life-threatening illnesses

that were rejected for efficiency.

What's your feeling
in general about AI?

I mean, technology is going
to find its way into everything,

so to stand against it,
you become a fossil.

But we have got to continue
to invest into

the human components of things
because as we saw,

you can't just automate everything,
right?

Somebody has to be looking
and watching over.

CHANTING: No denying!
People are dying!

I myself have had a terrible
experience with American healthcare,

despite having
one of the better plans.

Both my mom and I have had to battle

and fight tooth and nail
AI claim denials.

It's really insidious,

the way that AI is infecting
every aspect of our society,

but specifically
something so important and crucial,

such as healthcare that already
shouldn't be for profit.

They're using AI to maximise

the profits that they can make
off of their clients.

Stop denials with AI!
How many people have to die?

In this site, incredible strength
of feeling from the protesters.

You can understand, too.

If you've been
personally affected by this

or if someone in your family has,

I understand why this is
such an emotional moment.

But, you know, I also think that
someone was murdered here,

someone with a family, with children,

and I think there's
a bit of mental gymnastics

going on with these protesters

where they sort of conveniently
manage to forget that fact.

Free Luigi! Free Luigi!

In situations like these
where emotions are high,

it's sometimes easy
to turn on technology,

especially when it feels like it's
something we don't fully understand

and can't easily control.

AI didn't create the perceived
problems with US healthcare,

but it has supercharged them,

and to understand how,

it helps to know
what an AI algorithm actually is.

I think the words "algorithm" and
"AI" get thrown around quite a lot.

It gets quite confusing,

so I thought I would explain the
difference between the two

in the most New York way possible -

by imagining that you are
opening up a new hot dog stand.

Now, you've got
a couple of options here.

You could use an algorithm.

An algorithm is a series of steps
for completing a task.

In the case of a hot dog stand,
the task is to sell hot dogs.

The inputs are the sausages,
the onions.

The algorithm is the instructions -

cook 100 hot dogs a day,

sell them for $4 each,

stay open till 11 on Friday -

and the output
is hopefully a tidy profit.

CASH REGISTER CHIMES

With this traditional type
of algorithm,

everything is spelled out in advance.

It means it's very precise,
it's very reliable,

but it's also completely inflexible.

Now imagine that you've got a stand
that is run by an AI algorithm.

So, this time,
you don't tell it what to do.

You just give it the inputs -
the buns, the sausages, the onions -

and you say,
"The only thing I care about

"is how much money you make."

Now, at first,
AI is just going to watch.

It's just going to collect data.

It's going to hunt for patterns like
when people are buying the most,

where has the most footfall,

how that changes with the weather.

And then after a while,

it's going to start suggesting things
that you hadn't even thought of,

like pitch up outside a dog park
on Sundays

or start selling vegetarian sausages

or people tip more
when the cart smells like onions.

But the thing is, you didn't
tell the AI any of those rules.

It discovered them for itself.

AI algorithms are capable
of crunching

huge quantities of data

and analysing
complex patterns of behaviour,

all in order to make
your hot dog stand profitable.

And that is
the really key difference here.

For an AI algorithm,
you don't need to spell out

every possible scenario in advance.

You just define the goal
and let the AI learn for itself.

Just because an algorithm uses AI
doesn't mean it's necessarily better,

but it will ruthlessly pursue
whatever goal it's set.

I wanted to find out more

about the specific algorithm
being used by UnitedHealthcare,

so I tracked down
the two investigative journalists

who were the first
to uncover the story.

Hey. Hey, how are you? How you doing?
Good to meet you.

I'm Hannah. Nice to meet you.
Lovely to meet you.

Hey. Bob. Lovely to meet you, Bob.
How are you doing? Good.

Casey Ross and Bob Herman

were nominated for a Pulitzer Prize
for their investigation.

How big has this story been for you
in terms of your journalistic career?

I think this is probably
the biggest story that I've done

in terms of the impact
that it's had.

I think that the reaction that
we got

and some of the fallout in terms of
lawsuits and the US Senate

validate that this was a story
that had an extraordinary impact.

What was the first spark of
the story?

This person in the nursing home
industry sent me an email,

and it was just
this visceral reaction that,

hey, health insurers are issuing
a lot of denials

and they're not telling us why.

The people that were in
this facility were getting removed

when they were still very sick
and were not ready to go home.

So, it was just a signal that, OK,

maybe we should ask
some more questions about this.

And what we found was
that this all centres around

an algorithm called nH Predict,

and that algorithm is used
on behalf of insurance companies

to reduce the amount of time
that people are in these facilities

and to control their cost of care.

How much sight do we have of
the actual algorithm

that's going on underneath this?

Yeah, so there are
a bunch of pieces of data

that they are feeding into a model -

the age of the person,
what was their primary diagnosis,

what other illnesses did they have.

It compares that patient
to other patients like them

in a database of 6 million patients.

And based on this comparison,

this is the amount of care
you should get.

Once that prediction is made,

basically, a date is circled
on the calendar,

And this is the date
that they're trained

to push these people toward.

This is the report
that's produced by the algorithm,

and it all boils down to
this prediction,

which is that
the estimated length of stay

of the patient in this case
is 16.6 days.

.6? Yes. So, it's like to the hour?

It predicts it
down to the decimal point.

You can't predict when somebody's
going to be fine to the hour.

No, and it's the type of information

that only an AI algorithm
could give you.

It doesn't take into account
a lot of things about these people.

Every healthcare journey
is different.

Every injury, every recovery,

everything that comes up
in the course of your care

causes all sorts of things to happen
that can't possibly be predicted.

Literally, they're boiling down
these people to numbers.

"Oh, you're not just a number."
Yes, you are, and there it is.

There is a counterargument
to all of this, right,

in that people are quite accustomed

to artificial intelligence
that compares you to other people -

like, you know,
the recommendation algorithms

that you get on Netflix
or Spotify or Amazon.

It's like, people like you did this.
Therefore, do you want this?

Yeah, and I think
the common experience,

or at least my experience,
is that most of the time,

I think the algorithm is wrong,
it's way off.

Just because I watched
this cooking show

does not mean I want
to watch this other show.

But if you're talking about me
in the hospital

or after a serious injury
and it gets that wrong,

well, I have a much bigger problem
with that.

Casey and Bob's damning reporting
concluded in 2024,

throwing UnitedHealthcare
into the spotlight...

..just months before
Brian Thompson was shot dead.

Can you remember when you first heard
about Luigi Mangione?

I remember the morning
that it happened.

We started getting texts from people
and we're like,

"Holy crap.
Is this actually for real?"

We don't have any indication

that this individual was inspired
by our reporting,

but nonetheless, as a reporter,
you're stunned, you're shocked.

You don't ever want
any of your reporting

to inspire somebody to act that way.

Luigi Mangione is not someone

you would expect to take on
the role of outlaw vigilante.

He studied at the prestigious
University of Pennsylvania,

one of America's Ivy League.

EMCEE: Luigi Nicholas Mangione.
Luigi.

You can see in this video
of his high school graduation

that Luigi already stood out.

He's giving the valedictorian speech,
which means he's being celebrated

as the most academically successful
student in his year.

Family, friends, faculty
and fellow students, good afternoon.

He was clearly
a popular, confident young man.

We spoke to a few of his classmates
who didn't want to go on the record

because they have been continually
hounded by the press ever since,

which you can understand.

And everyone, almost to a point,
say that Luigi was smart,

he was helpful, he was kind,
he was friendly,

he was a popular guy
from a good family

who was also really well educated.

But then I discovered
a remarkable detail about Luigi,

one that's been all but overlooked.

He did computer
and information science as his major,

but then he also then got a master's
degree from here in computer science

with a concentration
in artificial intelligence.

One of the modules
that he was taking in here

was exactly the stuff
that sits behind this algorithm.

It's called data structures
and algorithms.

Very meaty,
very mathematical algorithms course

about data structures
and computer science - it's tough.

And the thing is, Luigi wasn't just
doing well in this course.

He was doing so well that he
was appointed as an assistant tutor.

Look at this photo.
I mean, he looks so confident.

He was basically teaching
other students

who were the same age as him.

This is stuff that he knew
incredibly well

and, by the sounds of it,
was also very, very good at.

Most of Luigi's friends have
so far refused to go on the record,

but back in the UK, I tracked down
one person who was willing to talk.

Gurwinder Bhogal is a British blogger

who writes about the impact
of technology on society.

Hi. Hi, Hannah. How are you doing?
Nice to meet you.

Lovely to meet you.
Thank you for this.

Luigi was a subscriber
to Gurwinder's blog,

and the two of them
talked about their shared concerns.

When I spoke to Luigi,
he was in Japan,

and he was remarking on how sort of
lonely it felt in a lot of places.

The streets were empty
because everything's just automated.

He was quite concerned
about mass automation

and the knock-on effects that
this might be having on society.

You can live your entire life
without leaving your house.

You can do your shopping online,

you can do your banking online,
you can do your dating online.

You can even have
a full relationship online.

There's less human connection

and AI is going to make this
a lot worse.

I mean, he himself probably
had no problem making friends.

He was a very
charismatic individual,

but I think he was
worried about other people

and maybe how they were
kind of gradually being lost,

the connections that we have
that keep society together.

And he was worried that
all of that is unravelling.

Did you talk about healthcare
as well?

There was only one brief exchange
that we had.

Luigi made a passing remark
about how I was lucky

because we had the NHS in the UK,
free healthcare.

How did he seem to you?

He actually seemed quite cheerful,

someone who did not seem
particularly pessimistic,

although some of the issues that
he raised were quite pessimistic.

But his general demeanour
was actually the opposite of that.

He said that he wanted me to focus
less on the problems

and more on the solutions.

He would ask me, "What's
the practical takeaway of this?"

He wasn't just
interested in moaning.

He wanted to find real solutions.

Since his arrest,
the Luigi Mangione story

had taken on a life of its own...

Luigi Mangione, the internet's
favourite hot Italian sausage.

..and social media had exploded
with memes

about the alleged killer.

Luigi,
you're a brave Italian stallion

whose actions ignited
a national dialogue

about the USA's
crappy healthcare system.

If you feel the same way about me
as I feel about you,

please do not deny,
delay or defend your love.

There was also
no shortage of theories

about what happened and why.

There is a note which was
supposedly found in his backpack

when he was arrested,

and the press have decided
to call this his manifesto.

In the document,

he appeared to talk about
the planning of the alleged crime,

but then moved on
to health insurance.

There's one bit here that says,

"Frankly, these parasites
had it coming."

Then he specifically
name checks United,

and then he says,

"These...indecipherable...have
simply gotten too powerful,

"and they continue to abuse
our country for immense profit

"because the American public has
allowed them to get away with it."

The other thing that
the internet is obsessed with

is his own personal story.

It's how he ended up feeling so
strongly about this particular issue.

I've got his X profile.

In the middle of his banner,
he's got this X-ray...

..and we know that
he had a spinal problem.

We know that he had surgery for it.

And that's...I mean,
that's pretty extreme, that surgery.

But what is interesting
is that there's no evidence

that he was actually a client
of UnitedHealthcare.

And so far,
nobody's found anything to prove

that he had a personal
insurance claim denied even.

As humans, we understand
being let down by other humans.

Real doctors often
don't have all the answers,

but we forgive them
for their very human flaws.

But when it comes to technology,

we have
very little tolerance for error.

The flipside is also true -

when AI promises
extraordinary medical possibilities,

it's all too easy for us
to believe in the impossible.

Just a few miles
across the state line,

I've been invited to meet
a young tech entrepreneur

who is selling a seemingly incredible
new medical breakthrough...

Hey! Hey!

..using AI.

How are you? Yeah, very good.

25-year-old Kian Sadeghi

runs a tech start-up
called Nucleus Genomics.

Are you sequencing
the entire genome? The entire thing.

The whole thing? The whole thing.

His company uses AI algorithms
to analyse the DNA

of his customers' future children
like never before.

These are nitrogen tanks, are they?
Yes.

How many embryos do you reckon
there are in here? Like millions?

If all the samples here
were embryos,

yes, there would be millions, yes.

Millions of potential new humans.
Yeah.

Kian uses AI
to map each embryo's DNA,

comparing it against
huge DNA databases

to try to predict
a baby's future risk of disease.

Some diseases can be predicted
with certainty.

For others, he can say how people
with similar DNA turned out.

When I talk to a couple, they say,
"My grandfather had Alzheimer's.

"I want to do anything I can

"to make sure
my son doesn't have Alzheimer's."

And then I think, well, genetics
obviously can help with that.

So, I think more people
are going to use IVF

and they're going to be using
genetic optimisation technology

to basically pick their child.

Figuring out whether a baby will
develop Alzheimer's later in life

is a best guess, not a diagnosis.

But the company goes further -
helping parents choose an embryo

based on eye colour,
height or even IQ.

Welcome to the Nucleus office.
Thank you.

Kian had already raised $32 million
from investors.

By the way,
if you ever see how many tabs I have

on the right side of the screen,

I don't think you should ever
show that on the camera

cos it is actually so insane.

I think it also slows down.
I think it uses up your RAM.

Anyways, let me show you this.

He just launched
a glossy new ad campaign.

I think it is. OK, let's go.

Nucleus Embryo is
for couples doing IVF

to uncover the full genetic profile
of each embryo

in one intuitive platform.

Every parent deserves
the power to decide

what possibility feels right
for their family.

Some people don't think
you should have this choice.

You're not afraid of controversy,
are you?

KIAN LAUGHS

No, no, no. It's not... It's...

You know, if you were doing IVF
and you had five embryos,

would you want to pick
your future baby randomly,

or would you ask the doctor
for more information on each,

especially if you have a family
history of Alzheimer's, of cancer?

It's my right
to know this information.

It's my choice to say I want a baby
with lower disease risk,

a baby that's slightly taller

or even with a specific eye colour,
etc.

It's their right, it's their choice.

I think what you're doing
where people have

genetic predispositions
for particular diseases,

especially when they're preventable
or treatable, it is amazing.

What I don't understand
is why you would include

something like IQ and eye colour,

cos it's so controversial
and it's a little bit eugenics-y.

You know, you are implicitly saying
that taller is better.

You are reinforcing the ideas
of preferences

that some people
are more valuable than others.

No, not at all.

I think parents have the right to
choose across their embryos, right?

If they want a baby with
a lower disease risk, for example,

or if they want a baby
that's shorter or taller,

that's absolutely their right
to choose.

This is designer babies, though.
Bluntly, this is designer babies.

No, it's not designer babies
at all, actually.

How is it not? It's not at all.

If a parent wants to give
their child the best start in life,

that is like a parent doing the most
basic, in my mind, and human thing.

If a parent...

The moment a child's born,

they will run multiple tests
on a baby

to make sure the baby's healthy.

They'll give it vaccines,
for example,

to make sure
they don't get diseases.

This is just another tool in the
toolkit that helps parents do that.

But there was something else
I was worried about.

Even the best scientists
don't fully understand

how genes and environment combine
to make us who we are.

I was worried that all of
this complexity was being overlooked

in favour of
tantalisingly simple AI predictions.

Do you think that
the technology is mature enough,

accurate enough, capable enough,

to be giving people this illusion
that they have control?

The platform, I think,
does an excellent job

showing the uncertainty
and showing the fact that, again,

DNA is not destiny
and DNA will never be destiny.

People can have
certain just genetic dispositions,

but there's
the whole thing called life.

There's environment,
there's how you're nurtured,

how you're raised, nutrition, etc.

We cannot possibly reduce human life
to just a DNA strand.

Testing for physical characteristics
is banned in the UK,

but there's nothing to stop British
couples travelling to the US for it,

as long as they're willing to pay
the approximate $40,000 price tag.

KNOCK ON DOOR

Hi! How are you doing?
Hello, welcome. Thank you.

Dragos and Laura live in London,

and were at the start of their
IVF journey with Kian's company.

These are beautiful.
Are these your babies?

Yes, our two children. Two boys.

So, your two boys,
did you have them naturally?

Yes, we did.

So, why not naturally for the third?

Because Dragos was kind of afraid.

We are... I'm 40, he's 43.

When you're over 43,

it was clear that the chances
of having issues are very high.

If this wasn't possible,

would you still want
to have another baby?

Definitely not. Not? Not.

Because you don't want to
take chances.

Everybody knows how important
a healthy child is,

and I think everybody should
do the extra mile for that outcome.

Where are you at now, then,
in the process?

In less than two months,

we are going to New York
to start the IVF process.

And then they will test the egg.
They will take few cells.

They will zoom in on anything
that can happen.

And then once we have selected
the right embryo,

then we would go back to New York
for the implantation,

which takes only one or two days.

So, in theory, in a few months,
you could be pregnant.

Yes. Fingers crossed.

Thank you. Yeah.

We both want a healthy child,
a healthy embryo.

We are also hoping
she's going to be a girl.

SHE CHUCKLES

Is that the dream, a little girl?
Yes.

This was my dream all along,
to have three babies.

So, if a girl would come, would be
the perfect picture for myself.

When you picture your daughter

or your potential daughter
in your head,

what does she look like?

Beautiful.

Um... Brown eyes, brown hair.

Light skin. Big lips, full lips.

LAUGHING: I don't know.

So, she'll be looking like you.

BOTH LAUGH

Not like me!

Hopefully not bald...with a beard.

Some people worry about
the Nucleus stuff

because it effectively prioritises

some characteristics of the baby
over others.

For example, IQ, eye colour,
height is stuff that you can,

in theory, at least give
a probability to, right?

You can measure.

I would not go through this just
for the height and the eye colour,

but if height is
what we can find out now,

maybe it's important, maybe not.

But, you know, we'll take that
because it's on the table.

They give you a probability
of the IQ that he or she might have.

Um... But, yes.

And that's going to be one
you'll look at? Of course.

SHE LAUGHS

Of course we will.

Before Dragos and Laura
started their IVF cycle,

there was a first step -

getting their own DNA mapped
by Nucleus...

I don't know, let's see.

..to see if they were carriers
for any diseases

they could pass on to their baby.

The results of these genetic tests
had just come in.

The moment of truth.

Feels like unravelling a present.

You don't know
exactly what's inside.

So, our family summary.

We have 2,154 rare diseases tested

and one detected risk for children.

Fuchs endothelial corneal dystrophy.

Symptoms often begin at 50.

So, imagine our child
is born this year.

50 years from now,
I'm sure we'll have bionic eyes

that we can replace
hard-wired to our brain.

So I don't think
this is serious enough

for us to be worried about it.

Of all of the things
that could potentially be risky -

Alzheimer's, Parkinson's,
breast cancer...

Yeah, this is very... Low level.
Low level, mild.

"Don't worry. You and Laura
can still have healthy children."

So, I think this is important. Mm.

I think if you're having a baby
in your 40s,

I can completely understand

why you would want
as much information as possible.

If you are prone to worry,

why wouldn't you try and eliminate
the risks where you possibly can?

But at the same time,

I'm worried that
this technology in this way

gives the illusion of control
that you don't actually have.

Gives the illusion of certainty

and a prediction of the future
that doesn't really exist.

Over in the US,

Luigi's case had proceeded
through the courts.

Charged with murder
in the first degree,

killing as an act of terrorism and
criminal possession of a weapon,

he faced his plea hearing...

..stoking further outrage
on both sides.

CHANTING: One struggle, one fight!

Healthcare is a human right!

We have very disturbed people
who somehow think

that eliminating a father
who has two young children

over some cause
is somehow justified.

But on the other side of America,

another legal case was
quietly gaining its own momentum.

A class action lawsuit against
UnitedHealthcare's use of AI

had been launched
by one plucky public interest firm

called Clarkson Law.

They'd also launched suits
against other big insurers

for similar claims.

Lawyer Glenn Danas
is leading the charge.

We know for sure that
there is an algorithm of some kind

that is predicting how long you need
in rehabilitation,

and we also know that some people
think that wasn't long enough.

Is that fair? Yes.

I mean, from our perspective,
that's a vast understatement.

but that is in fact true, yes.

The length of stay value
is consistently too low,

and it seems highly unlikely
that this is an accident

because it's only ever
in one direction.

How does the legal case come in?

One way it's illegal

is because there are
different states like California

that require that it be a human
making decisions,

so that by law it cannot be
delegated to an AI, an algorithm,

anything other than
a human medical professional

exercising his or her professional
knowledge and specialty.

How many people are involved in this?
How many plaintiffs are there?

Because United has such
a large market share in America,

it's almost certainly in millions.

If you eventually win this case,

does that mean that
they have to compensate everybody

who held that type of insurance?

What we want, whether it's by
a settlement or by going to trial,

is to have all the people
who were denied

what was owed to them
to be paid for that,

and then to change these practices
going forward.

The Clarkson case
rests on the evidence

of former UnitedHealthcare customers
who have come forward to testify,

and in particular, those who
have survived to tell their tale.

Hey. Bill, this is Hannah.

Bill. Hi. Such a treat to meet you.

How are you doing? Very good,
thank you. Lovely to meet you.

One of those is
86-year-old Bill Hull.

How are you guys doing? Well,
as well as can be expected, I guess.

Your koi are quite the beasts.

Can I chuck them in?

So, tell me, how are you doing now?
How's your health at the moment?

About 70% of my heart is shot. Mm.

But the worst part has been the
paralysis that the stroke caused me.

I used to be very active. Mm.

We did this thing,
did all the bricking here.

Can't do any of that any more.

In June 2023,
he suffered a heart attack

on his way to a medical appointment.

There's a park bench,

big, long park bench
just outside the building.

I just slumped over, apparently.

Two medical technicians that knew
CPR laid me down on the bench

and they broke all my ribs,
and I understood later,

if you don't break ribs,
you ain't doing it right. Wow.

I was in the hospital for 25 days.

In intensive care?

Almost all of it was intensive.

I'm there, and I think on
the 22nd or 23rd day,

I got a notice that,
"You're to be released."

They said, "Well, we'll assign
a case manager to you."

She got ahold of me and said,
"I have talked with your doctors,

"and everyone recommends
that you go into skilled nursing."

A day later, she came back and said,
"Well, I don't know. They said no."

They would not approve it
and wouldn't give me a reason.

Clarkson Law argued
that this denial was likely

the result of
UnitedHealthcare's Predict algorithm.

They made you feel like
you needed to get out,

and yet they were giving you
no place to go.

They said,
"Where do you want to go?"

I said, "I guess I'm going to
go home."

"OK, we'll get you out of here."
Stuck me in the car.

What physical state
were you in at this point?

My wife was very worried

because she didn't know
if she could take care of me.

She's 85 years old.

She has macular degeneration,
has a hard time seeing.

She's using a walker, too,
like I am.

Anyway, I get home.

I was two and a half,
three days out of the hospital.

My daughter came over
and I was in bad shape,

I think slurring my words.

I was starting to have a stroke.

I could not move a finger,
couldn't move my arm,

couldn't move my foot.

You feel yourself
just literally dying.

That's terrifying.

Oh, worst thing
that ever happened to me.

Do you think that
you would have had the stroke

regardless of where you were?

Well, I think
I'd have had the stroke,

but I should have been in either a
skilled nursing or in the hospital.

I'd have gotten some attention
several hours before,

and what I ended up with

would've been much less than
what I ended up with.

Bill is now looked after
by his daughters, Lisa and Laura.

What's your take on
artificial intelligence

being used in your dad's healthcare?

Certainly, the whole time
he was in the hospital,

we assumed it was
doctors who worked for that hospital

who were making these decisions
and who were denying this care.

And it wasn't until he found
the class action lawsuit

that he believed
it was actually the AI system

that was being utilised
by UnitedHealthcare.

I guess if a human had been
making that decision,

you'd able to have a conversation
with them. You would think.

If an algorithm says it, you can
kind of wash your hands of it.

I feel like that's
what they're doing.

If they can make AI
do their dirty work,

I think they're very happy
to do that.

How do you feel about the murder of
Brian Thompson?

I think it's indicative of how
frustrated human beings can become

with huge corporations
like UnitedHealthcare.

But I 100% don't condone
what happened

and wouldn't condone
any action like that in the future,

but the anger and the upset is real
and it's justified.

The thing is, I'm not anti-AI, right.
I'm not anti-algorithms.

I think that there is
an incredible amount of waste

that happens
when humans make decisions.

I think
there's incredible efficiencies

and therefore better care
that you can provide people

when you carefully introduce
these kind of systems.

The problem is that
once you start automating stuff,

it all comes down to
exactly what that algorithm

was designed to care about.

You know,
was it designed to care about

improving the outcomes
for the patients in the long term,

or was it designed to minimise
the amount of money

that is spent on
caring for the patient?

And those are two things that are...

..often at odds with one another.

But the problems with algorithms
go deeper.

Here is one thing
that you should know about AI.

Sometimes people call it a black box,
and there is a good reason for that.

You have to imagine that at one end,

you're putting in some information -
your input -

and at the other end,
you get some results - your output.

Now, the question is,

what is going on in the middle
inside of the algorithm?

Now, there is nothing magical
going on here. There's no voodoo.

It's just loads and loads
and loads of calculations.

But these things are so big
and unwieldy

that it becomes impossible
to follow one thread

from one end
all the way through to the other.

And that means
that artificial intelligence

is sometimes finding patterns
that we cannot see,

are unable to check
and might not like.

This problem was highlighted in 2019

by a researcher
called Ziad Obermeyer.

He looked at an AI algorithm
being used in hospitals

to identify patients
most in need of care

and offer them help
on a special programme.

But he discovered that the patients
the algorithm selected

were disproportionately white.

These algorithms should have been
a great use case for AI,

but unfortunately, a design choice
in building those algorithms

made them biased.

Obermeyer couldn't see inside the AI,

so he worked backwards
from the results

and found that the algorithm
had used a short cut.

It wasn't finding
the sickest patients,

it was finding the ones who'd had
the most money spent on their care.

Black patients have
less money spent on them

by our healthcare system today

because of barriers to access
and because of discrimination.

And that means that
the AI saw that fact clearly.

It predicted the cost accurately.

But instead of undoing
that inequality,

it reinforced it
and enshrined it in policy.

Even though the objective of
the algorithm was good,

the outcome led to discrimination,

and until Obermeyer, no human had
been there to spot the difference.

We may never be able to see
inside the algorithm

used by UnitedHealthcare
to assess claims.

The closest we can get is by talking
to those who worked alongside it.

And until now,
very few company insiders

have ever spoken
on the record about this AI,

but one former employee
had agreed to meet me.

Lovely to meet you.
Would you like something to drink?

Thank you so much.
There's ice and everything.

Amber Lynch was a care coordinator,

responsible for entering patient data
into the algorithm

and ensuring
they were discharged on time.

So, tell me your background, then.
What did you train as?

My background is
I'm an occupational therapist.

20 years I was working in clinics,
in hospitals.

I actually did home health,
so I went to their home.

I did it all, basically.

And then I had the opportunity
to go nonclinical.

So, talk me through the process.

Well, when I got a new case,

I was given doctor's notes,
I was given admission notes,

I was given therapy,
evaluations and assessments.

I would put all of that information
into a programme called the Predict,

and it would generate
a recommended discharge date.

Having an estimate of when somebody

is going to no longer need
specialist nursing care,

there's nothing wrong
with that in theory, though, right?

Patients always do best
when they're at home.

But if they're not safe at home,
you have to do it in rehab.

How often was that number of days

about the right ballpark
as you saw it?

Probably 20% of the time. Really?

Mm-hm. 20, 25%, yeah.

Was it sometimes more,
sometimes less?

Generally, it would be
three to four days under.

They liked to say
that the Predict took in

6 million patients' experiences.

I still believe that as humans,
we made better decisions.

You've got all this experience.

Can you use
your human judgment instead

if you saw that it was wrong?

Wouldn't that be great?

SHE LAUGHS

Unfortunately, no.

The expectation was
I would stay within 3%

of that estimated discharge date,

but by the time I stopped working,
it was down to 1%.

1%? 1%.

That means this patient who is
very sick and can't get out of bed,

guess what -
ten days, you're out of here.

That's not OK.

I always made it very clear
that I was just the messenger.

I do not make the decisions.

I had members' families
scream at me.

How did that feel?

Awful, because I wanted to just say,
"No, I 100% agree with you.

"I don't think that your mother
should be discharged right now."

But I wasn't allowed to.

They expect you
to meet these certain goals,

and the problem is
if you don't meet them,

then you're costing the company
too much money,

and it goes against you
as a care coordinator.

So, you as an employee
have repercussions if you don't...

Yes. If you don't stick with it.

..if these patients
don't get discharged

within 1% of the date
that the algorithm says.

I never met that metric. Mm-hm.

That was part of the reason
that I was let go.

It was all about the dollar
and I hated that.

Amber really cares about
her patients.

I mean, that is so obvious.

She, like, feels personally affronted
by what she was being asked to do.

Where this has fallen down has been
in the fact that it is so inflexible.

And it's this -
the idea that we might ultimately

give up control from humans
to machines -

that gets to the heart of
the feelings of injustice

that can arise
when technology clashes

with human pain and suffering.

HORNS BEEP

In New York, jury selection
in Luigi Mangione's trial

is scheduled to begin
later this year...

CHANTING: One struggle, one fight!
Healthcare is a human right!

..and it's bound to elicit
yet more controversy on all sides.

His supporters hope those jurors
will deliver a radical verdict.

We have a feature
in the American justice system

called jury nullification,
where if a jury believes

that a not-guilty verdict would be
the best delivery of justice,

they can deliver
a not-guilty verdict

regardless of whether they think the
person actually committed the crime.

CHANTING: The people united
will never be defeated!

My time in the US had made me
think about our fate in the UK.

AI is already being used in the NHS,
but it's being done with caution

and crucially,
with human supervision -

a tool to support humans,
not to override them.

And for me,
that is when AI is at its best -

not something to be feared,

but something to be carefully
incorporated into our lives.

AI could achieve
extraordinary things,

but this is a revolution that
has to happen with us, not to us.

To discover more about AI
and how it can shape our future,

go to...

..or scan the QR code
on the screen now.