All language subtitles for We.Need.to.Talk.About.A.I.2020.1080p.AMZN.WEB-DL.DDP5.1.H.264-TEPES

If we build artificial
general intelligence,

that'll be the biggest event

in the history of life on Earth.

Alien intelligence inside
a computer system that

has control over the
planet, the day it arrives...

And science fiction books and movies

have set it up in advance.

What's the problem?

I think you know what the problem is

just as well as I do.

What are you talking about, HAL?

This mission is too important

for me to allow you to jeopardize it.

I don't know what you're
talking about, HAL.

HAL, a super-intelligent computer.

We'd call it AI today.

My character, Dave, was the astronaut

in one of the most
prophetic films ever made

about the threat posed to humanity

by artificial intelligence.

HAL was science fiction.

But now, AI is all around us.

Artificial intelligence
isn't the future.

AI is already here.

This week a robot ran for mayor

in a small Japanese town.

The White House says

it is creating a new task force

to focus on artificial intelligence.

Google has already announced plans

to invest more in AI research.

IBM sending an artificial
intelligence robot into space.

Because of my small
role in the conversation,

I understand that we're on a journey,

one that starts with narrow AI.

Machines that perform specific tasks

better than we can.

But the AI of science fiction

is artificial general intelligence,

machines that can do everything

better than us.

Narrow AI is really

like an idiot savant to the extreme,

where it can do something,

like multiply numbers really fast,

way better than any human.

But it can't do
anything else whatsoever.

But AGI, artificial
general intelligence,

which doesn't exist yet, is instead

intelligence that can do everything

as well as humans can.

So what happens if we do create AGI?

Science fiction authors have warned us

it could be dangerous.

And they're not the only ones.

I think people should
be concerned about it.

AI is a fundamental risk

to the existence of human civilization.

And I don't think people
fully appreciate that.

I do think we have to worry about it.

I don't think it's inherent

as we create super intelligence

that it will necessarily always

have the same goals in mind that we do.

I think the development

of full artificial intelligence

could spell the end of the human race.

...the end of the human race.

Tomorrow, if the headline
said an artificial intelligence

has been manufactured
that's as intelligent

or more intelligent than human beings,

people would argue about
it, but I don't think

they'd be surprised,
because science fiction

has helped us imagine all kinds

of unimaginable things.

Experts are divided about the impact

AI will have on our future,

and whether or not Hollywood
is helping us plan for it.

Our dialogue has really
been hijacked by Hollywood

because The Terminator
makes a better blockbuster

than AI being good,

or AI being neutral,

or AI being confused.

Technology here in AI

is no different. It's a tool.

I like to think of it as a fancy pencil.

And so what pictures we draw with it,

that's up to us as a society.

We don't have to do what AI tells us.

We tell AI what to do.

I think Hollywood films

do help the conversation.

Someone needs to be thinking

ahead of what the consequences are.

But just as we can't really imagine

what the actual science of
the future is going to be like,

I don't think we've begun

to really think about all
of the possible futures,

which logically spring
out of this technology.

Artificial intelligence has
the power to change society.

A growing chorus of criticism

is highlighting the dangers

of handing control to machines.

There's gonna be a lot of change coming.

The larger long-term concern

is that humanity will
be sort of shunted aside.

This is something
Stanley Kubrick and others

were worried about 50 years ago, right?

It's happening.

How do we gauge the urgency
of this conversation?

If people saw on radar right now

that an alien spaceship
was approaching Earth

and it was 25 years away,

we would be mobilized to prepare

for that alien's arrival
25 years from now.

But that's exactly the
situation that we're in

with artificial intelligence.

Could be 25 years, could be 50 years,

but an alien intelligence will arrive

and we should be prepared.

Well, gentlemen, meet Tobor.

An electronic simulacrum of a man.

Gosh. Gee willikers.

Even though much work remains
before he is completed,

he is already a sentient being.

Back in 1956 when the term

artificial intelligence
first came about,

the original goals of
artificial intelligence

were human-level intelligence.

He does look almost kind, doesn't he?

Over time that's proved
to be really difficult.

I think, eventually,
we will have human-level

intelligence from machines.

But it may be a few hundred
years. It's gonna take a while.

And so people are getting
a little over-excited

about the future capabilities.

And now, Elektro, I
command you to walk back.

Back.

Almost every generation,

people have got so enthusiastic.

They watch 2001,

they fall in love with HAL, they think,

"Yeah, give me six weeks.
I'll get this sorted."

It doesn't happen and
everyone gets upset.

So what's changed in AI research?

The difference with the AI we have today

and the reason AI suddenly
took this big leap forward

is the Internet. That is their world.

Tonight the information superhighway,

and one of its main thoroughfares,

an online network called Internet.

In 1981, only 213 computers

were hooked to the Internet.

As the new year begins, an estimated

two-and-a-half
million computers

will be on the network.

AIs need to eat.

Just like sheep need grass,

AIs need data.

And the great prairies of
data are on the Internet.

That's what it is.

They said, "You know what?

"Let's just let them loose out there."

And suddenly, the AIs came to life.

Imagine thousands of talking robots

able to move as one unit,

taking on crime, fires,

and natural disasters.

Artificial intelligence platform Amper

has created the first
album entirely composed

and produced by artificial intelligence.

Next week, Christie's will
be the first auction house

to offer artwork created
by artificial intelligence.

Driverless cars, said to be

in our
not-so-distant future.

So even though we don't have AGI

or human-level intelligence yet,

are we ready to give autonomy

to the machines we do have?

Should self-driving cars
make ethical decisions?

That is the growing
debate as the technology

moves closer towards mainstream reality.

This idea of whether or not

we can embed ethics into machines,

whether or not they
are automated machines

or autonomous machines,

I'm not sure we've got that figured out.

If you have a self-driving car,

it's gonna have to
make ethical decisions.

A classic one people have
been thinking a lot about is...

I don't know if you've
heard of trolley problems.

The trolley problem.The trolley problem.

The trolley problem.The trolley problem.

In the trolley problem,

whether or not a driverless car

is going to save the driver or run over

a group of school
children crossing the road.

Will the car go right
and hit the old lady

or will it go left and
kill the four young people?

Either kill five people
standing on a bus stop

or kill yourself.

It's an ethical decision

that has to be made in the moment,

too fast for any person to get in there.

So it's the software
in your self-driving car

that's gonna make that ethical decision.

Who decides that?

Smart society,

society that values human life,

should make a decision that

killing one person is preferable
to killing five people.

Of course, it does raise the
question who's gonna buy that car.

Or, you know, are you gonna
buy the car where actually,

it'll swerve and you'll be fine,

but they're goners.

There are hundreds of
those kinds of questions

all throughout society.

And so, ethical philosophers
battle back and forth.

My peeve with the philosophers is

they're sucking the
oxygen out of the room.

There's this metaphor that I have of

you have a car driving
away from a wedding,

and the car is driving, and
behind it there are cans.

And the metal cans are
clanking and making noise.

That car is science

and technology driving
forward decisively.

And the metal cans clanking

are the philosophers
making a lot of noise.

The world is changing in such a way

that the question of
us building machines

that could become more
intelligent than we are

is more of a reality
than a fiction right now.

Don't we have to ask questions now,

before the machines get smarter than us?

I mean, what if they
achieve consciousness?

How far away is that?

How close we are to a conscious machine?

That is the question that brings

philosophers and scientists
to blows, I think.

You are dead center

of the greatest scientific
event in the history of man.

If you've created a conscious machine,

it's not the history of man.

That's the history of gods.

The problem with
consciousness is hardly anybody

can define exactly what it is.

Is it consciousness
that makes us special?

Human?

A way of thinking about consciousness

which a lot of people will agree with...

You'll never find something
that appeals to everyone.

...is it's not just flashing
lights, but someone's home.

So if you have a...

If you bought one of those little

simple chess-playing machines,

it might beat you at chess.

There are lots of lights flashing.

But there's nobody in when it wins,

going "I won."

All right? Whereas your dog

is a rubbish chess player,

but you look at the dog and you think,

"Someone's actually in.
It's not just a mechanism."

People wonder whether
general intelligence

requires consciousness.

It might partly depend on what level of
general intelligence you're talking about.

A mouse has some kind
of general intelligence.

My guess is, if you've
got a machine which has

general intelligence at the
level of, say, a mouse or beyond,

it's gonna be a good contender
for having consciousness.

We're not there yet.

We're not confronted by humanoid robots

that are smarter than we are.

We're in this curious ethical position

of not knowing where and
how consciousness emerges,

and we're building
increasingly complicated minds.

We won't know whether
or not they're conscious,

but we might feel they are.

The easiest example is if you programmed

into a computer everything
you knew about joy,

every reference in literature,

every definition,
every chemical compound

that's involved with
joy in the human brain,

and you put all of that
information in a computer,

you could argue that it understood joy.

The question is, had it ever felt joy?

Did HAL have emotions?

Will AIs ever feel the way we do?

It's very hard to get
machines to do the things

that we don't think about,
that we're not conscious of.

Things like detecting
that that person over there

is really unhappy with the way that...

Something about what I'm doing.

And they're sending a subtle signal

and I probably should
change what I'm doing.

We don't know how to build
machines that can actually be

tuned in to those kinds
of tiny little social cues.

There's this notion that maybe
there is something magical

about brains,

about the wetware of having

an information processing system

made of meat, right?

And that whatever we do in our machines,

however competent they begin to seem,

they never really will be intelligent

in the way that we
experience ourselves to be.

There's really no basis for that.

This silly idea you can't be intelligent

unless you're made of meat.

From my perspective, as a physicist,

that's just carbon chauvinism.

Those machines are made of exactly

the same kind of elementary particles

that my brain is made out of.

Intelligence is just a certain...

And consciousness is just

a certain kind of
information processing.

Maybe we're not so special after all.

Could our carbon-based brains

be replicated with silicon?

We're trying to build a
computer-generated baby

that we can teach like a real baby,

with the potential for
general intelligence.

To do that, you really have
to build a computer brain.

I wanted to build a
simple toy brain model

in an embodied system

which you could interact
with face-to-face.

And I happen to have an infant at home,

a real baby.

And I scanned the baby

so I could build a 3D model out of her

and then use her face

as basically the
embodiment of the system.

If a brain which is made out of cells,

and it's got blood
pumping, is able to think,

it just may be possible
that a computer can think

if the information is moving
in the same sort of process.

A lot of artificial
intelligence at the moment today

is really about advanced
pattern recognition.

It's not about killer robots

that are gonna take over
the world, et cetera.

So this is Baby X. She's
running live on my computer.

So as I move my hand
around, make a loud noise,

she'll get a fright.

We can kind of zoom in.

So she basically can see
me and hear me, you know.

Hey, sweetheart.

She's not copying my smile,

she's responding to my smile.

So she's responding to me

and we're really
concerned with the question

of how will we actually interact

with artificial
intelligence in the future.

When I'm talking to Baby X, I'm
deliberately modulating my voice

and deliberately doing
big facial expressions.

I'm going "Ooh," all this stuff

because I'm getting her attention.

Ooh.

And now that I've got her attention,

I can teach something.

Okay, so here's Baby X,

and this is... She's been
learning to read words.

So here's her first word book.

So let's see what she
can see. Turn to a page...

And here we go. Let's see what she...

What's this, Baby?
What's this? What's this?

What's this? Sheep.

Good girl. Now see if she
knows what the word is.

Okay. Baby, look over here.

Off you go. What's this? What's this?

Sheep.Good girl.

Let's try her on something else.

Okay.

- Let's try on that.
- What's this?

Baby, Baby, over here. What's this?

Baby. Look at me. Look
at me. What's this?

Baby, over here. Over here.

Puppy.Good girl.

That's what she's just read.

Ooh.

The most intelligent system

that we're aware of is a human.

So using the human as a template,

as the embodiment of Baby X,

is probably the best way to try

to achieve general intelligence.

She seems so human.

Is this the child who takes our hand

and leads us to HAL?

Whether it's in a real
world or a virtual world,

I do think embodiment

is gonna be very crucial here.

Because I think that
general intelligence

is tied up with being
able to act in the world.

You have to have something,

arms, legs, eyes,

something that you can move around

in order to harvest new information.

So I think to get systems
that really understand

what they're talking about,

you will need systems that can act

and intervene in the world,

and possibly systems that have something

almost like a social life.

For me, thinking and
feeling will come together.

And they will come together

because of the importance of embodiment.

The stories that we tell ourselves

open up arenas for discussion,

and they also help
us think about things.

Ava!

The most dominant story...

Go back to your room.

...is AI is scary.

There's a bad guy, there's a good guy...

That is a non-productive storyline.

Stop!

Ava, I said stop!

Whoa! Whoa! Whoa!

The reality is AI is
remarkably useful to us.

Robot surgeons are increasingly
used in NHS hospitals.

They don't get tired,
their hands don't tremble,

and their patients
make a faster recovery.

It alerted me that there
was one animal in heat.

She was ready to be bred,
and three animals that

may have a potential health problem.

A Facebook program can
detect suicidal behavior

by analyzing user information,

and Facebook CEO Mark
Zuckerberg believes

this is just the beginning.

Here is someone who
has visual impairment

being able to see because of AI.

Those are the benefits of what AI does.

Diagnose cancer earlier.

Predict heart disease.

But the revolution has only just begun.

AI is.

It's happening. And so it is a question

of what we do with it, how we build it.

That's the only question
that matters. And I think,

we potentially are at
risk of ruining that

by having this knee-jerk reaction

that AI is bad and
it's coming to get us.

So just like my astronaut,

we're on a journey.

Narrow AI is already here.

But how far are we from AGI?

And how fast are we going?

Every five years,

computers are getting ten times faster.

At the moment, we don't have yet

little devices, computational devices,

that can compute as
much as a human brain.

But every five years,

computers are getting ten times cheaper,

which means that soon we
are going to have that.

And once we have that,
then 50 years later,

if the trend doesn't break,

we should have for the same price

little computational
devices that can compute

as much as all ten billion brains

of humankind together.

It's catching up.

What happens if it passes us by?

Now a robot that's been
developed by a Japanese scientist

is so fast, it can win

the rock-paper-scissors
game against a human

every single time.

The presumption has to be that

when an artificial intelligence

reaches human intelligence,

it will very quickly
exceed human intelligence,

and once that happens,

it doesn't even need us anymore

to make it smarter. It
can make itself smarter.

You create AIs which are better than us

at a lot of things.

And one of the things they get better at

is programming computers.

So it reprograms itself.

And it is now better than it was,

which means it's now better
than it was at reprogramming.

So it doesn't get better like that.

It gets better like that.

And we get left behind

as it keeps getting faster,

evolving at an exponential rate.

That's this event that
a lot of researchers

refer to as the singularity.

And it will get beyond

where we are competitive very quickly.

So think about someone not
having a PhD in one area,

but having it in every conceivable area.

You can't figure out what they're doing.

They can do things no
one else is capable of.

Just imagine what it would be like

to be in dialogue with another person

running on a time course

that was a million
times faster than yours.

So for every second you had to think

about what you were going to do

or what you were going to say,

it had two weeks to
think about this, right?

Um, or a billion times faster,

so if for every second you had to think,

it it's high-stakes
poker or some negotiation,

every second you are
thinking, it has 32 years

to think about what
its next move will be.

There's no way to win in that game.

The singularity,

the moment the machines surpass us.

Inevitable? Not everyone

at the forefront of AI thinks so.

Some of my friends who are
worried about the future

think that we're gonna
get to a singularity

where the AI systems are able

to reprogram themselves
and make themselves

better and better and better.

Maybe in the long-distant future.

But our present AI systems

are very, very narrow at what they do.

We can't build a system today

that can understand a computer program

at the level that a brand-new
student in programming

understands in four or five weeks.

And by the way,

people have been working on this problem

for over 40 years.

It's not because people haven't tried.

It's because we don't
have any clue how to do it.

So to imagine it's suddenly
gonna jump out of nothing

I think is completely wrong.

There's a negative narrative about AI.

Brilliant people like Elon Musk

use very religious imagery.

With artificial intelligence,
we are summoning the demon.

You know? You know
all those stories where

there's the guy with the
pentagram and holy water

and he's like, "Yeah," he's
sure he can control the demon.

Doesn't work out.

You know, people like me

who've spent their entire career

working on AI are literally
hurt by that, right?

It's a personal attack almost.

I tend to think about
what's gonna happen

in the next ten years,
in the next 25 years.

Beyond 25,

it's so difficult to predict

that it becomes very much a speculation.

But I must warn you,

I've been specifically programmed

not to take over the world.

All of a sudden, for the first time

in the history of the field,
you have some people saying,

"The reason we don't have to worry

"about any kind of

"existential risk to humanity

"from super-intelligent machines

"is because it's impossible.

"We're gonna fail."

I... I find this completely bizarre.

Stuart Russell cowrote the textbook
many AI students study with.

He understands the momentum
driving our AI future.

But given the rate of investment in AI,

the number of incredibly smart people

who are working on these problems,

and the enormous incentive,

to achieve general purpose intelligence,

back of the envelope calculation,

the value is about
10,000 trillion dollars.

Because it would revolutionize
the economy of the world.

With that scale of incentive,

I think betting against human ingenuity

just seems incredibly foolish.

China has a big vested interest

in being recognized as,
kind of, an AI superpower.

And they're investing what I believe is

upwards of $150 billion

the next five to ten years.

There have been some news flashes

that the government
will start ramping up

its investment in AI.

Every company in the Fortune 500,

whether it's IBM, Microsoft,

or they're outside like Kroger,

is using artificial intelligence

to gain customers.

I don't think that people
should underestimate

the types of innovations going on.

And what this world is gonna look like

in three to five years from now,

I think people are gonna have
a hard time predicting it.

I know many people who work

in AI and they would tell you,

"Look, I'm going to
work and I have doubts.

"I'm working on creating this machine

"that can treat cancer,

"extend our life expectancy,

"make our societies better,
more just, more equal.

"But it could also be
the end of humanity."

The end of humanity?

What is our plan as a species?

We don't have one.

We don't have a plan
and we don't realize

it's necessary to have a plan.

A lot of the AI researchers

would love for our science fiction books

and movies to be more balanced,

to show the good uses of AI.

All of the bad things we tend to do,

you play them out

to the apocalyptic
level in science fiction

and then you show them to people

and it gives you food for thought.

People need to have that warning signal

in the back of their minds.

I think it's really important

to distinguish between
science on the one hand,

and science fiction on the other hand.

I don't know what's gonna happen

a thousand years from now,

but as somebody in the trenches,

as somebody who's building AI systems,

who's been doing so for more
than a quarter of a century,

I feel like I have my hand on the pulse

of what's gonna happen next,

in the next five, ten, 20 years,

much more so than these folks

who ultimately are not
building AI systems.

We wanna build the
technology, think about it.

We wanna see how people react to it

and build the next
generation and make it better.

Skepticism about whether this poses

any danger to us seems peculiar.

Even if we've given it
goals that we approve of,

it may form instrumental
goals that we can't foresee,

that are not aligned with our interest.

And that's the goal alignment problem.

Goal alignment.

Exactly how my battle with HAL began.

The goal alignment
problem with AI is where

you set it a goal

and then, the way that it does this goal

turns out to be at odds
with what you really wanted.

The classic AI alignment
problem in film...

Hello. HAL, do you read me?

...is HAL in2001.

HAL, do you read me?

HAL is given a clear
goal.Do you read me, HAL?

Make the mission succeed.
But to make it succeed,

he's gonna have to make

some of the people on the mission die.

Affirmative, Dave. I read you.

Open the pod bay doors, HAL.

I'm sorry, Dave.

I'm afraid I can't do that.

"I'm sorry, Dave, I can't do that,"

is actually a real genius illustration

of what the real threat
is from super-intelligence.

It's not malice. It's competence.

HAL was really competent

and had goals that just
didn't align with Dave's.

HAL, I won't argue with you anymore.

Open the doors!

Dave, this conversation

can serve no purpose anymore. Goodbye.

HAL?

HAL?

HAL?

HAL!

HAL and I wanted the same thing.

Just not in the same way.

And it can be dangerous
when goals don't align.

In Kubrick's film, there was a solution.

When HAL tried to kill me,

I just switched him off.

People say, "Well,
just switch them off."

But you can't just switch them off.

Because they already thought of that.

They're super-intelligent.

For a machine convinced of its goal,

even something as simple
as "fetch the coffee."

It's obvious to any
sufficiently intelligent machine

that it can't fetch the
coffee if it's switched off.

Give it a goal, you've given the machine

an incentive to defend itself.

It's sort of a psychopathic behavior

of the single-minded
fixation on some objective.

For example, if you wanted
to try and develop a system

that might generate world peace,

then maybe the most efficient,

and

speedy way to achieve that

is simply to annihilate all the humans.

Just get rid of the people,

they would stop being in conflict.

Then there would be world peace.

Personally, I have very little time

for these parlor discussions.

We have to build these systems,

experiment with them,

see where they go awry and fix them.

We have so many opportunities
to use AI for the good,

why do we focus on the fear?

We are building them.

Baby X is an example.

Will she be a good AI?

I think what's critical in this

is that machines are accountable,

and you can look at them

and see what's making them tick.

And the best way to understand
something is to build it.

Baby X has a virtual physiology.

Has pain circuits and
it has joy circuits.

She has parameters which
are actually representing

her internal chemical
state, if you like.

You know, if she's stressed,
there's high cortisol.

If she's feeling good, high dopamine,

high serotonin, that type of thing.

This is all part of, you know,

adding extra human
factors into computing.

Now, because she's got these
virtual emotional systems,

I can do things like...

I'm gonna avoid her or hide over here.

So she's looking for where I've gone.

So we should start seeing
virtual cortisol building up.

Aww!

I've turned off the audio
for a reason, but anyway...

I'll put her out of
her misery. Okay. Sorry.

It's okay, sweetheart. Don't
cry. It's okay. It's okay.

Good girl. Okay. Yeah.

So what's happening is she's responding

to my facial expressions and my voice.

But how is chemical state

represented in Baby X?

It's literally a mathematical parameter.

So there's levels,
different levels of dopamine,

but they're represented by numbers.

The map of her virtual
neurochemical state

is probably the closest

that we can get to felt experience.

So if we can build a computer model

that we can change the settings
of and the parameters of,

give it different experiences,

and have it interact with us

like we interact with other people,

maybe this is a way to explore

consciousness in a new way.

If she does become conscious,

what sort of life will she want?

An AI, if it's conscious,

then it's like your child.

Does it have rights?

Last year, European
lawmakers passed a resolution

to grant legal status
to electronic persons.

That move's now being considered

by the European Commission.

Machines with rights.

Are we ready for that?

I think it raises moral
and ethical problems,

which we really

are most content to ignore.

What is it that we really want?

We want a creature, a being, that we can

interact with in some way,

that is enough like us
that we can relate to it,

but is dispensable or disposable.

The first and most notable one,

I think, was inMetropolis.

The Maria doppelganger
that was created by

the scientist simulated a sexy female.

And this goes back to
our penchant for slavery

down through the ages.

Let's get people that we can,

in a way, dehumanize

and we can do what we want with them.

And so, I think, we're just, you know,

finding a new way to do slavery.

Would it be right for us
to have them as our slaves?

This all becomes
slightly weird ethically

if we happen to, by
choice or by accident,

build conscious slaves.

That seems like a morally
questionable thing to do,

if not an outright evil thing to do.

Artificial intelligence
is making its way

into the global sex market.

A California company will soon roll out

a new incredibly lifelike sex robot

run by artificial intelligence.

One application that does trouble me

is the rise of sex robots

that are shaped in the
image of real women.

I am already taking over the world

one bedroom at a time.

If they are conscious,

capable of sentience,

of experiencing pain and suffering,

and they were being
subject to degrading, cruel

or other forms of utterly
unacceptable treatment,

the danger is that
we'd become desensitized

to what's morally
acceptable and what's not.

If she's conscious, she
should be able to say no.

Right?

Shouldn't she have that power?

Aren't these the questions
we need to ask now?

What do we want it to mean to be human?

All other species on
Earth, up until now,

have not had the luxury

to answer what they want it to mean

to be a mouse or to be an earthworm.

But we have now reached
this tipping point

where our understanding
of science and technology

has reached the point that
we can completely transform

what it means to be us.

What will it mean to be human?

If machines become more,
will we become less?

The computer Deep Blue
has tonight triumphed over

the world chess champion Garry Kasparov.

It's the first time
a machine has defeated

a reigning world champion.

More on that showdown

on one of America's most beloved

game shows, Jeopardy!

Two human champions going

brain-to-brain with a
super-computer named Watson.

What isThe Last Judgment?

Correct. Go again. Watson.

Who is Jean Valjean? Correct.

A Google computer program has

once again beaten a
world champion in Go.

Go, an ancient Chinese board game,

is one of the most
complex games ever devised.

It's another milestone in
the rush to develop machines

that are smarter than humans.

We have this air of superiority,

we as humans.

So we kind of got used to be
the smartest kid on the block.

And I think that this
is no longer the case.

And we know that because we rely on AI

to help us to make many
important decisions.

Humans are irrational
in many very bad ways.

We're prejudiced, we
start wars, we kill people.

If you want to maximize the chances

of people surviving overall,

you should essentially
give more and more

decision-making powers to computers.

Maybe machines would
do a better job than us.

Maybe.

I am thrilled and honored to be here

at the United Nations.

The future is here. It's
just not evenly distributed.

AI could help efficiently distribute

the world's existing resources,

like food and energy.

I am here to help
humanity create the future.

The one super-intelligent scenario

that is provocative to think about

is what I call the benevolent dictator.

Where you have this
super-intelligence which runs Earth,

guided by strict ethical
principles that try

to help humanity flourish.

Would everybody love that?

If there's no disease,

no poverty, no crime...

Many people would
probably greatly prefer it

to their lot today.

But I can imagine that
some might still lament

that we humans just
don't have more freedom

to determine our own course. And, um...

It's a tough one. It really is. We...

I don't think most people would wanna be

like zoo animals, with no...

On the other hand...

Yeah.

I don't have any glib answers

to these really tough questions.

That's why I think
it's just so important

that we as a global community

really discuss them now.

Machines that can read your face.

Recognize emotional cues.

Closer to a brain process of a human...

Robots imbued with life...

An emotional computer...

From this moment on,
regardless of whether you think

we're just gonna get
stuck with narrow AI

or we'll get general AI
or get super-intelligent AI

or conscious super-intelligent...

Wherever you think we're gonna get to,

I think any stage along that

will change who we are.

Up and at 'em, Mr. J.

The question you have to ask is,

is AI gonna do something for you

or something to you?

I was dreaming about sleeping.

If we build AI systems

that do everything we want,

we kind of hand over our world to them.

So that kind of enfeeblement
is kind of irreversible.

There is an existential risk
that is real and serious.

If every single desire,

preference, need, has
been catered for...

Morning, Rosie. What's for breakfast?

...we don't notice that our capacity

to make decisions for ourselves

has been slowly eroded.

This is George Jetson,

signing off for the entire day.

There's no going back from there. Why?

Because AI can make such decisions

with higher accuracy

than whatever a human being can achieve.

I had a conversation with
an engineer, and he said,

"Look, the safest way
of landing a plane today

"is to have the pilot leave the deck

"and throw away the key,

"because the safest way

"is to use an autopilot, an AI."

We cannot manage the complexity

of the world by ourselves.

Our form of intelligence,

as amazing as it is, is severely flawed.

And by adding an additional helper,

which is artificial intelligence,

we stand the ability
to potentially improve

our cognition dramatically.

The most cutting-edge
thing about Hannes Sjoblad

isn't the phone in his hand,

it's the microchip in his hand.

What if you could type something

just by thinking about it?

Sounds like sci-fi,

but it may be close to reality.

In as little as five years,

super-smart people could
be walking down the street,

men and women who've paid

to increase their intelligence.

Linking brains to
computers by using implants.

What if we could upload a digital copy

of our memories and
consciousness to a point

that human beings could achieve

a sort of digital immortality?

Just last year,

for the first time in world history,

a memory was recorded.

We can actually take recorded memory

and insert it back into an animal,

and it remembers.

Technology is a part of us.

It's a reflection of who we are.

That is the narrative we
should be thinking about.

We think all that exists is
everything we can imagine.

When in reality, we can imagine

a very small subset of
what's potentially possible.

The real interesting stuff in the world

is what sits beyond our imagination.

When these massive systems emerge,

as a species, I think we get an F

for our ability to predict
what they're going to become.

Technology has always changed us.

We evolve with it.

How far are we prepared to go this time?

There was a time when Aristotle

lamented the creation
of the written language

because he said this would
destroy people's memories

and it would be a terrible thing.

And yet today, I carry
my brain in my pocket,

in the form of my smartphone,

and that's changed the
way I go through the world.

I think we're absolutely
merging with our technology.

Absolutely. I would... I would
use the term "co-evolving."

We're evolving it to match our needs

and we are changing to match
what it can provide to us.

So instead of being
replaced by the machines,

we're going to merge with them?

I have a number of friends
who have bionic arms.

Are they cyborgs?

In a way, they're on that path.

I have people who have implants

in their ears. Are they AIs?

Well, we're in a way on that path

of augmenting human bodies.

Human potential lies

in this co-evolutionary relationship.

A lot of people are trying to hold onto

the things that make
them special and unique,

which is not the correct frame.

Let's create more.

Rewrite our intelligence,
improve everything

we're capable of. Emotions,
imagination, creativity.

Everything that makes us human.

There's already a symbiotic relationship

between us and a machine
that is enhancing who we are.

And... And there is a
whole movement, you know,

the trans-human movement,

where basically

people really bet bigtime

on the notion that

this machine-human integration

is going to redefine who we are.

This is only gonna grow.

There's gonna be a learning
process that gets us

to that endgame where we
live with these devices

that become smarter and smarter.

We learn how to adapt ourselves

around them, and they adapt to us.

And I hope we become
more human in the process.

It's hard to understand how merging

with the machines

can make us more human.

But is merging our
best chance of survival?

We humans are limited

to how many neurons we
can fit in our brain.

And there are other species
that have even smaller brains.

When you look at bees and fish moving,

it looks like a single organism.

And biologists actually refer
to it as a super-organism.

That is why birds flock,

and bees swarm, and fish school.

They make decisions that
are in the best interest

of the whole population

by thinking together
as this larger mind.

And we humans could do
the same thing by forming

a hive mind connecting
us together with AI.

So the humans and the software together

become this super-intelligence,

enabling us to make the
best decisions for us all.

When I first started looking
at this, it sounded scary to me.

But then I realized it's scarier

for an alien intelligence

to emerge that has
nothing to do with humans

than for us humans to connect together

and become a super-intelligence.

That's the direction that I work on,

because I see it as,

in some sense, the only alternative

to humans ultimately
being replaced by AI.

Can Baby X help us
understand the machines

so we can work together?

Human and intelligent machine
cooperation will define

the next era of history.

You probably do have to create
a virtual empathetic machine.

Empathy is a lot about connecting with

other people's struggles.

The more connection you have,

the more communication that you have,

the safer that makes it.

So at a very basic level,

there's some sort of
symbiotic relationship.

By humanizing this technology,

you're actually increasing the channels

for communication and cooperation.

AI working in a human world

has to be about humans.

The future's what we make it.

If we can't envision a future

that we really, really want,

we're probably not gonna get one.

Stanley Kubrick tried to imagine

what the future might look like.

Now that that future's approaching,

who is in charge of
what it will look like?

Who controls it? Google.

The military. Tech giant Facebook...

Samsung... Facebook, Microsoft, Amazon.

The tide has definitely turned

in favor of the corporate world.

The US military, and I
suspect this is true worldwide,

cannot hold a candle to the
AI that's being developed

in corporate companies.

If corporations,

not nations, hold the power,

what can we do in a capitalist system

to ensure AI is developed safely?

If you mix AI research with capitalism,

on the one hand, capitalism
will make it run faster.

That's why we're getting
lots of research done.

But there is a basic problem.

There's gonna be a pressure

to be first one to the
market, not the safest one.

This car was in autopilot mode

when it crashed in Northern California,

killing the driver.

Artificial intelligence may have

more power to crash the
markets than anything else

because humans aren't
prepared to stop them.

A chatbot named Tay.

She was designed to
speak like a teenager

and learn by interacting online.

But in a matter of
hours, Tay was corrupted

into a Hitler-supporting sexbot.

This kind of talk that somehow
in the context of capitalism,

AI is gonna be misused,
really frustrates me.

Whenever we develop technology,

whether it's, you know, coal-burning
plants, whether it's cars,

there's always that incentive
to move ahead quickly,

and there's always
the regulatory regime,

part of the government's
job to balance that.

AI is no different here.

Whenever we have technological advances,

we need to temper them

with appropriate rules and regulations.

Duh!

Until there's a major disaster,

corporations are gonna be saying

"We don't need to worry
about these things.

"Trust us, we're the experts.

"We have humanity's
best interests at heart."

What AI is really about
is it's a new set of rules

for companies in terms of
how to be more competitive,

how to drive more efficiency,
and how to drive growth,

and creative new business models and...

Once corporate scientists

start to dominate the debate,

it's all over, you've lost.

You know, they're paid
to deny the existence

of risks and negative outcomes.

At the moment, almost all AI research

is about making human lives

longer, healthier and easier.

Because the big companies,

they want to sell you AIs

that are useful for you,

and so you are only going to buy

those that help you.

The problem of AI in
a capitalist paradigm

is it's always gonna be cheaper

just to get the product out
there when it does the job

rather than do it safely.

Right now, the way it
works, you have a conference

with 10,000 researchers,
all of them working

on developing more capable
AI, except, like, 50 people

who have a tiny workshop
talking about safety

and controlling what
everyone else is doing.

No one has a working safety mechanism.

And no one even has a prototype for it.

In the
not-so-distant future,

you will have all kinds of AIs

that don't just do
what humans tell them,

but they will invent their own goals

and their own experiments,

and they will transcend humankind.

As of today,

I would say it's completely unethical

to try to develop such
a machine and release it.

We only get one chance to get it right.

You screw up, everything
possibly is lost.

What happens if we get it wrong?

The thing that scares
me the most is this AI

will become the product of its parents,

in the same way that you are and I am.

Let's ask who those parents are.

Right?

Chances are they're either
gonna be military parents

or they're gonna be business parents.

If they're business parents,
they're asking the AI

to become something

that can make a lot of money.

So they're teaching it greed.

Military parents are
gonna teach it to kill.

Truly superhuman intelligence

is so much... Is so super.

It's so much better than we are

that the first country
or company to get there

has the winner-take-all
advantage.

Because there's such an
arms race in AI globally,

and that's both literally,
in terms of building

weapons systems, but also figuratively

with folks like Putin
and the Chinese saying

whoever dominates AI is
gonna dominate the economy

in the 21st century.

I don't think we have a
choice to put the brakes on,

to stop developing and researching.

We can build the technology,
we can do the research,

and imbue it with our values,

but realistically, totalitarian regimes

will imbue the technology
with their values.

Israel, Russia, the US and China

are all contestants

in what's being described
as a new global arms race.

All countries, whether
they admit it or not,

are going to develop
lethal autonomous weapons.

And is that really a
capability that we want

in the hands of a corporate nation state

as opposed to a nation state?

The reality is if Facebook or Google

wanted to start an army,

they would have technologically
superior capabilities

than any other nation on the planet.

Some AI scientists are so concerned,

they produced this dramatized video

to warn of the risks.

Your kids probably have
one of these, right?

Not quite. We produced

the Slaughterbot video to illustrate

the risk of autonomous weapons.

That skill is all AI.

It's flying itself.

There are people within the AI community

who wish I would shut up.

People who think that
making the video is

a little bit treasonous.

Just like any mobile device these days,

it has cameras and sensors.

And just like your phones
and social media apps,

it does facial recognition.

Up until now, the
arms races tended to be

bigger, more powerful, more expensive,

faster jets that cost more.

AI means that you could have

very small, cheap weapons

that are very efficient at doing

one very simple job, like

finding you in a crowd and killing you.

This is how it works.

Every single major
advance in human technology

has always been weaponized.

And the idea of weaponized
AI, I think, is terrifying.

Trained as a team, they can penetrate

buildings, cars, trains,

evade people, bullets...
They cannot be stopped.

There are lots of
weapons that kill people.

Machine guns can kill people,

but they require a human
being to carry and point them.

With autonomous weapons,
you don't need a human

to carry them or supervise them.

You just program and then off they go.

So you can create the effect of an army

carrying a million machine guns,

with two guys and a truck.

Putin said that the
first person to develop

strong artificial intelligence
will rule the world.

When world leaders use terms

like "rule the world,"

I think you gotta pay attention to it.

Now, that is an air strike
of surgical precision.

I've been campaigning against
these weapons since 2007,

and there's three
main sets of arguments.

There's the moral
arguments, there's arguments

about not complying
with the laws of war,

and there are arguments
about how they will change

global security of the planet.

We need new regulations.

We need to really think
about it now, yesterday.

Some of the world's leaders
in artificial intelligence

are urging the United
Nations to ban the development

of autonomous weapons,

otherwise known as killer robots.

In the letter they say "once developed,

"they will permit armed
conflict to be fought

"at timescales faster than
humans can comprehend."

I'm a signatory to that letter.

My philosophy is,

and I hope the governments
of the world enforce this,

that as long as a
human life is involved,

then a human needs to be in the loop.

Anyone with a 3D printer and a computer

can have a lethal autonomous weapon.

If it's accessible to
everyone on the planet,

then compliance, for example,

would be nearly impossible to track.

What the hell are you doing? Stop!

You're shooting into thin air.

When you're imagining

waging war in the future,

where soldiers are robots,

and we pose no risk of
engagement to ourselves...

When war becomes just like a video game,

as it has incrementally already,

it's a real concern.

When we don't have skin in
the game, quite literally...

...when all of this is
being run from a terminal

thousands of miles away
from the battlefield...

...that could lower
the bar for waging war

to a point where we would recognize

that we have become monsters.

Who are you?

Hey! Hey!

Having been a former fighter pilot,

I have also seen people make mistakes

and kill innocent people

who should not have been killed.

One day in the future,
it will be more reliable

to have a machine do
it than a human do it.

The idea of robots

who can kill humans on their own

has been a science
fiction staple for decades.

There are a number of these
systems around already,

that work autonomously.

But that's called supervised autonomy

because somebody's there
to switch it on and off.

What's only been seen in Hollywood

could soon be coming to
a battlefield near you.

The Pentagon investing
billions of dollars

to develop autonomous weapons,

machines that could one
day kill on their own.

But the Pentagon insists
humans will decide to strike.

Once you get AI into military hardware,

you've got to make
it make the decisions,

otherwise there's no point in having it.

One of the benefits of AI
is that it's quicker than us.

Once we're in a proper
fight and that switch

that keeps the human in the loop,

we better switch it off,
because if you don't, you lose.

These will make the battlefield

too fast for humans.

If there's no human involved at all,

if command and control
have gone over to autonomy,

we could trigger a war
and it could be finished

in 20 minutes with massive devastation,

and nobody knew about
it before it started.

So who decides what we
should do in the future

now?

From Big Bang till
now, billions of years.

This is the only time a person can steer

the future of the whole humanity.

Maybe we should worry that
we humans aren't wise enough

to handle this much power.

Who is actually going
to make the choices,

and in the name of whom
are you actually speaking?

If we, the rest of
society, fail to actually

really join this conversation
in a meaningful way

and be part of the decision,

then we know whose values
are gonna go into these

super-intelligent beings.

Red Bull.Yes, Deon.

Red Bull.

It's going to be decided
by some tech nerds

who've had too much Red Bull to drink...

who I don't think have any particular

qualifications to speak

on behalf of all humanity.

Once you are in the
excitement of discovery...

Come on!

...you sort of turn a blind eye

to all these more moralistic questions,

like how is this going to
impact society as a whole?

And you do it because you can.

And the scientists are
pushing this forward

because they can see
the genuine benefits

that this will bring.

And they're also really, really excited.

I mean, you don't...
You don't go into science

because, well, you couldn't
be a football player,

"Hell, I'll be a scientist."

You go in because that's what
you love and it's really exciting.

It's alive!

It's alive! It's alive!

It's alive!

We're switching

from being creation to being God.

Now I know what it feels like to be God!

If you study theology,

one thing you learn is
that it is never the case

that God successfully manages

to control its creation.

We're not there yet,

but if we do create

artificial general intelligence,

can we maintain control?

If you create a self-aware AI,

the first thing it's gonna want

is independence.

At least, the one intelligence
that we all know is ourselves.

And we know that we would
not wanna be doing the bidding

of some other intelligence.

We have no reason to believe
that it will be friendly...

...or that it will have
goals and aspirations

that align with ours.

So how long before
there's a robot revolution

or an AI revolution?

We are in a world today where it is

absolutely a foot race

to go as fast as
possible toward creating

an artificial general intelligence

and there don't seem to be any brakes
being applied anywhere in sight.

It's crazy.

I think it's important to honestly

discuss such issues.

It might be a little too late to make

this conversation afterwards.

It's important to remember
that intelligence is not evil

and it is also not good.

Intelligence gives power.

We are more powerful than
tigers on this planet,

not because we have sharper claws

or bigger biceps, because we're smarter.

We do take the attitude that ants

matter a bit and fish matter
a bit, but humans matter more.

Why? Because we're more intelligent,

more sophisticated, more conscious.

Once you've got systems which
are more intelligent than us,

maybe they're simply gonna matter more.

As humans, we live in a
world of microorganisms.

They're life, but if a microorganism

becomes a nuisance,
we take an antibiotic

and we destroy billions of them at once.

If you were an artificial intelligence,

wouldn't you look at every little human

that has this very limited view

kind of like we look at microorganisms?

Maybe I would.

Maybe human insignificance

is the biggest threat of all.

Sometimes when I talk
about AI risk, people go,

"Shh! Max, don't talk like that.

"That's Luddite scaremongering."

But it's not scaremongering.

It's what we at MIT
call safety engineering.

I mean, think about it, before NASA
launched the Apollo 11 moon mission,

they systematically thought through

everything that could go wrong
when you put a bunch of people

on top of explosive fuel tanks
and launched them somewhere

where no one can help them.
Was that scaremongering?

No, that was precisely
the safety engineering

that guaranteed the
success of the mission.

That's precisely the strategy I
think we have to take with AGI.

Think through everything
that could go wrong

to make sure it goes right.

I am who I am because of the movie

A Space Odyssey.

Aged 14, it was an eye-opener for me.

That movie is the best
of all science fiction

with its predictions
for how AI would evolve.

But in reality,

if something is sufficiently
far in the future,

we can't imagine what
the world will be like,

so we actually can't
say anything sensible

about what the issues are.

Worrying about
super-intelligence safety

is a little like worrying about

how you're gonna install seat belts

in transporter devices.

Like in Star Trek,
where you can just zip

from one place to another.

Um, maybe there's some dangers to that,

so we should have seatbelts, right?

But it's of course very silly
because we don't know enough

about how transporter technology

might look to be able to say

whether seatbelts even apply.

To say that there won't
be a problem because

the unknowns are so vast,

that we don't even know
where the problem is gonna be,

that is crazy.

I think conversation is probably
the only way forward here.

At the moment, among ourselves,

and when the time comes,

with the emerging
artificial intelligences.

So as soon as there is anything

remotely in that ballpark,

we should start trying
to have a conversation

with it somehow.

Dog. Dog. Yeah.

One of the things we hope
to do with Baby X is to try

to get a better insight
into our own nature.

She's designed to learn autonomously.

She can be plugged into the Internet,

click on webpages by
herself and see what happens,

kind of learn how we learn.

If we make this super fantastic AI,

it may well help us become
better versions of ourselves,

help us evolve.

It will have societal effects.

Is it to be feared?

Or is it something to learn from?

So when might we be faced with machines

that are as smart, smarter than we are?

I think one of the ways to
think about the near future of AI

is what I would call the event horizon.

People have heard of black holes,

this super dense point where

if you get too close, you get sucked in.

There is a boundary. Once
you cross that boundary,

there is no way back.

What makes that dangerous is
you can't tell where it is.

And I think AI as a technology
is gonna be like that,

that there will be a point
where the logic of the money

and the power and the promise

will be such that once
we cross that boundary,

we will then be into that future.

There will be no way back,
politically, economically

or even in terms of what we want.

But we need to be worrying
about it before we reach

the event horizon, and
we don't know where it is.

I don't think it's gonna
happen any time soon.

The next few decades.

Twenty-five to 50 years.

A few hundred years.

It's over the next few decades.

Not in my lifetime.

Nobody knows. But it
doesn't really matter.

Whether it's years or 100
years, the problem is the same.

It's just as real and we'll
need all this time to solve it.

History is just full of
over-hyped tech predictions.

Like, where are those flying cars

that were supposed to be here, right?

But it's also full of the opposite,

when people said, "Never gonna happen."

I'm embarrassed as a physicist
that Ernest Rutherford,

one of the greatest nuclear
physicists of all time said

that nuclear energy
was never gonna happen.

And the very next day,

Leo Szilard invents the
nuclear chain reaction.

So if someone says
they are completely sure

something is never gonna happen,

even though it's consistent
with the laws of physics,

I'm like, "Yeah, right."

We're not there yet.

We haven't built a
super-intelligent AI.

And even if we stipulated
that it won't happen

shorter than 50 years from now, right?

So we have 50 years to think about this.

Well...

it's nowhere written that 50 years

is enough time to figure
out how to do this safely.

I actually think it's going to take

some extreme negative example

of the technology for
everyone to pull back

and realize that we've
got to get this thing

under control, we've
got to have constraints.

We are living in a science
fiction world and we're all,

um, in this big, kind of
social, tactical experiment,

experimenting on ourselves, recklessly,

like no doctor would
ever dream of doing.

We're injecting ourselves

with everything that we come up with

to see what happens.

Clearly, the thought process is...

You're creating learning robots.

Learning robots are
exactly what lead to Skynet.

Now I've got a choice, right? I can say,

"Don't worry, I'm not good at my job.

"You have nothing to fear
because I suck at this

"and I'm not gonna actually
accomplish anything."

Or I have to say, "You know,

"You're right, we're all gonna die,

"but it's gonna be sort of fun to watch,

"and it'd be good to be one of
the people in the front row."

So yeah, I don't have a
terrific answer to these things.

The truth, obviously,
is somewhere in between.

Can there be an in-between?

Or are we facing a new world order?

I have some colleagues who
are fine with AI taking over

and even causing human extinction,

as long as we feel

that those AIs are our worthy ancestors.

In the long run, humans
are not going to remain

the crown of creation.

But that's okay.

Because there is still beauty

and grandeur and greatness

in realizing that you are

a tiny part of

a much grander scheme which
is leading the universe

from lower complexity
towards higher complexity.

We could be rapidly
approaching a precipice

for the human race where we find

that our ultimate
destiny is to have been

the forebearer of the true

emergent, evolutionary,
intelligent sort of beings.

So we may just be here to give birth

to the child of this...
Of this machine...

Machine civilization.

Selfishly, as a human, of course,

I want humans to stick around.

But maybe that's a very
local, parochial point of view.

Maybe viewed from the point
of view of the universe,

it's a better universe with our...

With our super-intelligent descendants.

Our field is likely to succeed

in its goal to achieve
general purpose intelligence.

We now have the capability
to radically modify

the economic structure of the world,

the role of human beings in the world

and, potentially, human
control over the world.

The idea was that he is taken in

by godlike entities,

creatures of pure
energy and intelligence.

They put him in what you
could describe as a human zoo.

The end of the astronaut's journey.

Did he travel through time?

To the future?

Were those godlike entities
the descendants of HAL?

Artificial intelligence
that you can't switch off,

that keeps learning and learning until

they're running the zoo,

the universe,

everything.

Is that Kubrick's
most profound prophecy?

Maybe.

But take it from an old man.

If HAL is close to being amongst us,

we need to pay attention.

We need to talk about AI.

Super-intelligence.

Will it be conscious?

The trolley problem.

Goal alignment.

The singularity.

First to the market.

AI safety.

Lethal autonomous weapons.

Robot revolution.

The hive mind.

The transhuman movement.

Transcend humankind.

We don't have a plan, we don't realize

it's necessary to have a plan.

What do we want the
role of humans to be?