Original subtitles

Hi, I'm delighted to have with us here

today my old friend, professor Fei-Fei Li.

Fei-Fei is a professor of computer science at

Stanford University and also co-director of HAI;

the Human-Centered AI Institute.

Previously, she also was responsible for

AI at Google Cloud as a chief scientist for the division.

It's great to have you Fei-Fei.

Thank you, Andrew. Very happy to be here.

How long have we known each other? I've lost track.

Definitely more than a decade.

I've known your work before we even met and I came to

Stanford 2009 but we started talking 2007, so 15 years.

I actually still have very clear memories of

how stressful it was when collectively bunch of us;

me, Chris Manning, bunch of us.

We tried to figure out how to recruit

you to come to Stanford.

It wasn't hard.

I just needed to sort out my student's life,

but it's hard to resist Stanford

Really great to have you as a friend and colleague here.

Me too. It's been a long time

and we're very lucky to be the generation.

Seeing AI is great progress.

There was something about your background

I always found inspiring,

which is today people are entering AI from

all walks of life and sometimes people still wonder,

is AI a right path for me?

So I thought one of the most interesting parts

of your background was that

you actually started out

not studying computer science or AI,

but you started out studying physics and then had

this path to becoming one of

the most globally recognizable AI scientists.

How did you make that switch from physics to AI?

Well, that's a great question, Andrew,

especially both of us are passionate about

young people's future and they come into the world of AI.

The truth is, if I could enter

AI back then more than 20 years ago today,

anybody can enter AI because AI has become

such a prevailing and globally impactful technology.

But myself, maybe I was an accident.

I have always been a physics kid or a STEM kid.

I'm sure you were too,

but physics was my passion all the

way through middle school,

high school, college and I went

to Princeton and majored in physics.

One thing physics has taught me till today

is really the passion for asking big questions,

the passion for seeking no stars.

I was really having fun as

a physics student at Princeton.

One thing I did was reading

up stories and just writings of

great physicists of the 20th century and

just hear about what they think about the world,

especially people like Albert Einstein,

Roger Penrose, Erwin Schrodinger,

and it was really funny to

notice that many of the writings towards

the later half of the career of these great physicists

were not about just the atomic world

or the physical world,

but ponderings about

equally audacious questions like life,

like intelligence, like human conditions.

Schrodinger wrote this book, What is Life?

Roger Penrose wrote this book, Emperor's New Mind.

That really got me very curious

about the topic of intelligence.

One thing led to another during college time,

I did intern at the top of neuroscience labs

and especially vision-related, and I was like,

wow, this is just as audacious question to ask as

the beginning of the universe or what is matter made of?

That got me to switch from undergraduate degree

in physics to graduate degree in AI.

Even though I don't know about you during our time,

AI was a dirty word.

It was AI winter,

so it was more machine learning and computer

vision, and computational neuroscience.

Yeah, I know. Honestly, I think when I was an undergrad,

I was too busy writing code.

I just managed to blithely

ignore the AI winter and just kept on coding.

Yeah, well, I was too busy solving PDE equations.

Actually, do you have an audacious question now?

Yes, my audacious question is still intelligence.

I think since Alan Turing,

humanity has not fully understand what is

the fundamental computing principles behind intelligence.

Today we use the words AI,

we use the word AGI,

but at the end of the day,

I still dream of a set of simple equations or

simple principles that can

define the process of intelligence,

whether it's animal intelligence or machine intelligence.

This is similar to physics.

For example, many people

have drawn the analogy of flying.

Are we replicating birds

flying or are we building airplane?

A lot of people ask the question of

the relationship between AI and brain.

To me, whether we're

building a bird or

replicating a bird or building an airplane,

at the end of the day,

aerodynamics and physics that

govern the process of flying,

and I do believe one day we'll discover that.

I sometimes think about this

one learning algorithm hypothesis.

Could a lot of intelligence, maybe not all,

but a lot of it be explained by

one or a very simple machine learning principle?

It feels like we're still so far from cracking that nut.

But in the weekends when I have

spare time when I think about

learning algorithms and where they could go,

this is one of the things I'm

excited about. Just thinking about that.

I totally agree. I still feel like we're

pre-Newtonian if we're doing

physics analogy before Newton.

There has been great physics,

great physicists, a lot of phenomenology,

a lot of studies of how

the astral bodies move and all that.

But it was Newton who started to write very simple laws.

I think we are still going through

that very exciting coming of

age of AI as a basic science.

We're pre-Newton in my opinion.

It's really nice to hear you talk about how despite

machine learning and AI having come so far.

It still feels like

there are a lot more unanswered questions,

a lot more work to be done by maybe some of

the people joining the field

today than work that's already been done.

Absolutely. Let's calculate,

it's only 160 years about.

It's a very nascent field.

Modern physics and chemistry

and biology are all hundreds of years.

I think it is very exciting to be entering

the field of science of

intelligence and studying AI today.

Yeah, actually that's good. I think

I remember chatting with

the late Professor John McCarthy

who had coined the term artificial intelligence,

and boy, the field has changed since when he

conceived of it at

a workshop and came up with the term AI.

But maybe another ten years from now,

maybe someone watching this will come with a new set of

ideas and then we'll be saying,

boy, AI show us different than

what you and I thought it would be.

That's the exciting future to build towards.

Yeah, I'm sure Newton would have

not dreamed of Einstein.

Our evolution of science sometimes takes strides,

sometimes takes a while,

and I think we're absolutely

in exciting phase of AI right now.

It's interesting hearing you

paint this grand vision for AI.

Going back a little bit, there was one

of the piece of your background

that I found inspiring,

which is when you're just getting started,

I've heard you speak about how your physics student,

but not only that, you're also

running a laundromat to pay for school.

Just tell us more about that.

I came to this country,

to New Jersey actually when I was 15.

One thing great about being in New Jersey is

it was close to Princeton so I

often just take a weekend trip

with my parents and to admire

the place where Einstein spent most of

his career in the latter half of his life.

But with typical immigrant life and it was tough.

By the time I enter Princeton,

my parents didn't speak English

and one thing led to another.

It turns out running a dry cleaner

might be the best option for my family,

especially for me to lead

that business because it's a weekend business.

If it's a weekday business,

it would be hard for me to be a student.

It's actually, believe or not,

running a dry cleaning shop is very machine-heavy,

which is good for a STEM student like me.

We decided to open a dry cleaner shop

in a small town

in New Jersey called Parsippany in New Jersey.

It turned out we were physically

not too far from Bell Labs

and where lots of

early convolutional neural

network research was happening,

but I had no idea.

It's just summer intern at the ancient.

That is right, with Rob Shapiro.

With Michael Kearns was my mentor and Rob Shapiro,

inventor of those things creator algorithms.

You're encoding AI.

I was trying to [inaudible]

Only much later that I started interning.

Yeah. Then it was seven years.

I did that for the entire undergrad and most

of my grad school and I hire my parents. Yeah.

Yeah. Now, that's really inspiring.

I know you've been

bred into doing exactly where all your life.

I think the story of running

a laundromat to globally prominent computer scientists.

I hope that inspires

some people watching this that no matter where you are,

there's plenty of for everyone.

I don't even know this.

My high-school job was

office admin and so

to this day I remember doing a lot of photocopying.

The exciting part was using this shredder.

That was a glamorous one.

I was doing so much coffee in

high school I thought, boy, felony,

I could build a robot to do this for the coffee,

maybe I could do something.

Did you succeed?

Still working on it. When people

think about you and the work you've done,

one of the huge successes everyone thinks about

this ImageNet where help

establish early benchmark for computer vision.

It was really completely instrumental to

the modern rise of deep learning and computer vision.

One thing I bet not many people know about is

how you actually got started on ImageNet.

Tell us the origin story of ImageNet.

Yeah. Well Andrew,

that's a good question because a lot of

people see ImageNet as just labeling a ton of images.

But where we began was really

going after a Northstar

brings back my physics background.

When I enter grad school,

when did you enter grad year?

'97.

I was three years later that year, 2000.

That was a very exciting period because I was in

computer vision and computational neural science lab

of Pietro Purana and Christoph car at Caltech.

Leading up to that, there has been,

first of all, two things was very exciting.

One is that the world

of AI at that point wasn't called AI.

Computer vision or natural language processing

has founded Lingua Franca.

It's machine learning, statistical

modeling as a new tool has emerged.

I mean, it's been around.

I remember when the idea of

applying machine learning to computer vision,

that was a controversial thing.

I was the first generation of

graduate students who were embracing all the base net,

all the inference algorithms and all that.

That was one exciting happening.

A certainly exciting happening that

most people don't know and don't appreciate is

the couple of decades probably

one of them two or three decades of

incredible cognitive science

and cognitive neuroscience work

in the field of vision,

in the world of vision, human vision.

That has really established a couple

of really critical Northstar problems.

Just understanding of

human visual processing and human intelligence.

One of them is the recognition of

understanding of natural objects and natural things.

Because a lot of

the psychology and cognitive science work

is pointing to us.

That is an innately optimized,

whatever that word is.

Functionality and the ability of

human intelligence is more robust,

faster, and more nuanced than we had thought.

We even find neural correlates,

brain areas devoted to faces or places or body parts.

These two things lead to my PhD study of using

machine-learning methods to work

on real-world objects recognition.

But it becomes very painful very

quickly that we are coming banging

against one of the most continued to be

the most important challenge in

AI machine learning is the lack of generalizability.

You can design a beautiful model

or you want if you're overfitting the model.

I remember when it used be possible to publish

a computer vision and paper

showing it works on one image.

Exactly.

It's just the overfitting.

The models are not very expressive,

and we lack the data.

We also as a field was betting on making the variables

very rich by hand engineered features.

Remember, every variable carrying

a ton of semantic meaning,

but with hand engineered features.

Then towards the end of my PhD, my advisor,

Pietro and I start to look at each other and say,

well, boy, we need more data.

If we believe in

this North Star problem of object recognition,

and we look back at the tools we have,

mathematically speaking,

we're overfitting every model we're encountering.

We need to take a fresh look at this.

One thing led to another.

He and I decided we'll just do at that point.

We think it was a large-scale data project

called Caltech 101.

I remember the dataset.

I wrote papers using your Caltech 101 dataset way back.

You did, you and your early graduate student.

You have benefit a lot of researchers,

that Caltech 101 dataset.

That was me and my mom labeling images,

and a couple of undergrads,

but it was the early days of Internet.

Suddenly the availability of data was a new thing.

I remember Pietro still have

this super expensive digital camera.

I think it was Canon or something like

$6,000 walking around Caltech taking pictures.

But we're the Internet generation.

I go to Google image search,

I start to see these thousands and tens of thousands of

images and I tell Pietro, "Let's just download."

Of course it's not that easy to download.

One thing led to another.

We build this Caltech 101 dataset

of 101 object categories,

and about 30,000 pictures.

I think us really saying

that even though everyone's heard of ImageNet today,

even you took a couple of

iterations where you did

Caltech 101 and that was a success.

Lots of people used it for

even the early learnings from building Caltech 101.

They gave you the basis to build what turned

out to be an even bigger success.

Except that by the time I became an assistant professor,

we started to look at the problem.

I realized it's way bigger than we think.

Just mathematically speaking, Caltech 101 was

not sufficient to power the algorithms.

We decided to do image there.

That was the time people start to think

we're doing too much.

It's just too crazy,

the idea of downloading the entire Internet of images,

mapping out all the English nouns was a little bit.

I start to get a lot of push back.

I remember at one of the CVPR conference

when I presented the early idea of ImageNet,

a couple of researchers publicly questioned and said,

"If you cannot recognize one category of object,

let's say the chair you're sitting in,

how do you imagine or what's the use of a dataset of

22,000 classes of 15 million images."

In the end, that giant dataset unlocked a lot of

value for [inaudible] number

of researchers around the world.

I think it was the combination of betting on

the right North Star problem and the data that drives it.

It was a fun process.

To me when I think about that story,

it seems like one of

those examples where sometimes people

feel like they should only work on projects

without the huge thing at the first outset.

But I feel like for people working in machine learning,

if your first project is a

bit smaller, it's totally fine.

Have a good win. Use the learning

to build up to even bigger things,

and then sometimes you get

the ImageNet size win all of it.

But in the meantime,

I think it's also important to be

driven by an audacious goal though.

You can size your problem or size your project

as local milestones and so on along this journey,

but I also look at some of our current students.

They're so pure pressured by

this current climate of publishing nonstop.

It becomes more incremental papers to

just get into a publication for the sake of it.

I personally always push my students to ask the question,

what is the North Star that's driving you?

Yeah, that's true.

Myself when I do research over the years,

I've always pretty much done what I'm excited about,

where I want to try to push the view forward.

Doesn't have to listen to people.

Have to listen to people let them shape your opinion.

But in the end, I think

the best researchers let the world shape their opinion,

but in the end, drive things for using their own opinion.

Totally agree, yeah.

It's your own fire.

As a research program developed,

you've wound up taking your, let's say,

foundations in computer vision and neuroscience and apply

it to all sorts of topics

including your very visibly health care.

Looking at neuroscience applications.

We'd love to hear a bit more about that.

Yeah, happy to. I think

the evolution of my research in computer vision

also follows the evolution

of visual intelligence in animals.

There are two topics that truly excites me.

One is what is

a truly impactful application area

that would help human lives?

That's my health care work.

The other one is what is

vision at the end of the day about?

That brings me to

trying to close the loop between

perception and robotic learning.

On the healthcare side, one thing, Andrew,

there was a number that shocked me about

10 years ago when I met my long term collaborator,

Dr. Arnie Milstein at Stanford Medical School,

and that number is about a quarter of

a million Americans die of medical errors every year.

I had never imagined

a number being that high due to medical errors.

There are many reasons,

but we can rest

assure most of the reasons are not intentional.

These are errors of

unintended mistakes and solar, for example.

That's a mind boggling number.

It is.

It's been about 40,000 deaths

a year from automotive accidents.

It's just completely tragic and this is even [inaudible]

I was going to say that. I'm glad you brought it up.

Just one example.

One number within that mind-boggling number

is the number of hospital acquired

infection resulted fatality is more than 95,000.

That's 2.5 times than the death of car accidents.

In this particular case,

hospital acquired infection as a result of many things.

But in enlarge, lack of good hand hygiene practice.

If you look at WHO,

there has been a lot of protocols

about clinicians hand hygiene practice.

But in real health care delivery,

when things get busy and when the process

is tedious and when there's a lack of feedback system,

you still make a lot of mistakes.

Another tragic medical fact is that

more than $70 billion every year are spent

in full resulted injuries and fatalities.

Most of this happen to elderlies at home,

but also in the hospital rooms.

These are huge issues.

When Arnie and I got together back in 2012,

it was the height of self-driving car,

let's say not hype.

But what's the right word?

Excitement in Silicon Valley.

Then we look at the technology of smart sensing cameras,

lidars, whatever, smart sensors,

machine-learning algorithm, and holistic understanding

of a complex environment

with high-stakes for human lives.

I was looking at all that for self-driving car and

realized in healthcare delivery,

we have the same situation.

Much of the process,

the human behavior process of health care is in the dark.

If we could have smart sensors be it in patient rooms or

senior homes to help

our clinicians and patients to stay safer,

that will be amazing.

Arnie and I embarked on this,

what we call ambient intelligence research agenda.

But one thing I learned which probably will

lead to our other topics,

is as soon as you're applying

AI to real human conditions,

there's a lot of human issues

in addition to machine learning issues,

for example, privacy.

I remember reading some of

your papers with Arnie and found it really

interesting how you could build and deploy systems that

were relatively privacy preserving.

Yeah, well, thank you. Well,

the first iteration of that technology

is we use cameras that do not capture RGB information.

You've used a lot of that in self-driving car,

that depth cameras, for example.

There you preserve a lot of

privacy information just by

not seeing the faces and the identity of the people.

But what's really interesting over

the past decade is the changes of

technology is actually giving

us a bigger toolset for privacy,

preserved, a computing in this condition,

for example, on-device inference.

As the chips getting more and more powerful,

if you don't have to transmit any data

through the network and to the central server,

you help people better.

Federated learning, we know it's still early stage,

but that's another potential tool

for privacy, preserved computing.

Then differential privacy

and also encryption technologies.

We're starting to see that human demand,

privacy and other issues is driving actually a new wave

of machine learning technology

in ambient intelligence in health care.

Yeah, I've been encouraged to see

your real practical applications

of differential privacy that are actually real.

Federated Learning as you said,

Pray the PRRs little bit

ahead of the reality, but I think we'll get there.

But it's interesting how consumers

in the last several years have,

fortunately, gotten much more

knowledgeable about privacy and increasingly.

I think the public is

also making us to be better scientist.

Yeah, I think

ultimately people understand the AI hosts everyone,

including us, but holds everyone

accountable for really doing the right thing.

Yeah.

On that note, one of

the really interesting piece

of work you've been doing has

been leading several efforts to help educate legislators.

I hope governments, especially US government,

work through better laws and better regulation,

especially as it relates to AI.

That sounds a very important

and I suspect some days they'll be,

I would guess, somewhat frustrating work,

but we'd love to hear more about that.

Yeah, first of all,

I have to credit many, many people.

About four years ago,

I was actually finishing my sabbatical from Google time.

I was very privileged to work with so many businesses.

Enterprise developers, just a large

number and variety of

vertical industries are realizing AI's human impact.

That was one, meaning faculty leaders at

Stanford and also just our president, provost,

former President and former

provost all get together and realize there is

a historical role that Stanford needs

to play in advances of AI.

We were part of the birth place of AI.

A lot of work, our previous generation have done and

all of work you've done and some of the work

I've done lead to today's AI,

what is our historical opportunity and responsibility?

With that, we believe that

the next generation of AI education

and research and policy needs to be human centered.

Having established the humans center AI institute,

what we call HAI,

one of the work that really took

me outside of my comfort zone were aiming

expertise is really a deeper engagement

with policy thinkers and makers.

Because we're here in

Silicon Valley and there is a culture in

Silicon Valley is we just keep making

things and the law will catch up by itself.

But AI is impacting human lives and sometimes negatively

so rapidly that it is not good for any of us.

If we, the experts,

are not at the table with a policy thinkers and

makers to really try to make

this technology better for the people.

We're talking about fairness,

we're talking about privacy,

we also are talking about

the brain drain of AI to industry and

the concentration of data and compute

in a small number of technology companies.

All these are really part of the changes of our time.

Some are really exciting changes,

some have profoundly impact that we

cannot necessarily predicting yet.

One of the policy work that

Stanford AI has very proudly engaged in,

is we were one of the leading universities that lobbied

a bill called the

National AI Research Cloud Task Force Bill.

It changed the name from

research Cloud to research resource.

Now the bill's acronym is NAIR,

National AI Research Resource.

This bill is calling for a task force to put

together a road-map for America's public sector,

especially higher education,

and research sector to

increase their access to resource for AI compute and

AI data and really is aimed to rejuvenate

America's ecosystem in AI innovation and research.

I'm on the 12-person task-force

under Biden administration for this bill,

and we hope that's a piece of policy

that is not a regulatory policy,

it's more an incentive policy

to build and rejuvenate ecosystems.

I'm glad that you're doing this,

The Help Shape US Policy,

and making sure enough resources are

allocated to ensure healthy development of AI.

I feel like this is something that

every country needs at this point.

Yeah.

Just from the things that you are doing by yourself,

not to speak of the things that

the Global AI Community is doing,

there's just so much going on in AI right

now so many opportunities, so much excitement.

I found that for someone

getting started in machine learning for the first time,

sometimes there's so much going on and they can

almost feel a little bit overwhelming.

Totally.

What advice do you have for

someone getting started in machine learning?

Good question, Andrew, I'm sure you have great advice.

You're one of the world-known advocate

for AI machine learning education.

I do get this question a lot as well,

and one thing you're totally right

is AI really today feels different from our time.

Just for the record, you all are still on time.

That's true when we were starting in AI,

I love that exactly we're still part of this.

When we first started the entrance to

AI and machine learning was relatively narrow.

You almost have to start from computer science and go.

As a physics major,

I still had to wedge myself into

the computer science track or

electrical engineering track to get to AI.

But today, I actually think that there is many aspect of

AI that creates entry points

for people from all walks of life.

On the technical side,

I think it's obvious that there's

just incredible plethora of

resources out there on the Internet

from Coursera to YouTube,

to TikTok there's just so much that students

worldwide can learn about AI and

machine learning compared to

the time we began learning machine learning.

Also any campuses,

we're not talking about just college campuses

we're talking about high school campuses,

or even sometimes earlier,

we're starting to see

more available classes and resources.

I do encourage those

of the young people with a technical interest

and resource and opportunity to

embrace these resources because it's a lot of fun.

But having said that,

for those of you who are not

coming from a technical angle,

who still are passionate about AI,

whether it's the downstream application

or the creativity it engenders,

or the policy and social angle,

or important social problems,

whether it's digital economics or

the governance or history,

ethics, political sciences,

I do invite you to join us

because there is a lot of work to be done.

There's a lot of unknown questions, for example,

my colleague at HAI are trying to find answers

on how do you define our economy in the digital age?

What does it mean when robots,

or software, are

participating in the workflow more and more?

How do you measure our economy?

That's not AI coding question that is AI impact question.

We're looking at the incredible advances

of generative AI and there will be more.

What does that mean for creativity,

and to the creators from music to art, to writing?

I think there is a lot of concerns,

and I think it's rightfully so,

but in the meantime,

it takes people together to figure this

out and also to use this new tool.

In short, I just think it's a very exciting time,

and anybody with any walks of life,

as long as you are passionate about this,

there's a role to play.

That's really exciting when you talk about economics,

think about my conversations

with Professor Erik Brynjolfsson.

Impact of AI on the economy,

but from what you're saying and I agree,

it seems like no matter what your current interests are,

AI is such a general-purpose technology that

the combination of your current interest in

AI is often promising.

I find that even for learners

that may not yet have a specific interest,

if you find your way into AI,

start learning things,

often the interest will evolve and then you can

start to craft your own path.

Given way AI is today,

there's still so much room and so much need for

a lot more people to craft their own path,

to do this exciting work that I

think the world still needs a lot more of.

Totally agree.

One piece of work that you did

I thought was very cool was starting

a program initially called

SAILORS and then later AI4ALL,

which was really reaching out

to high school and even younger students,

to try to give them more opportunities in AI,

including people of all walks of life.

I'd love to hear more about that.

This is in the spirit of this conversation

is that was back in 2015.

There was starting to be a lot of excitement of AI,

but there was also starting to be this talk

about killer robot coming next door, terminators coming.

At that time Andrew,

I was the director of

Stanford AI Lab, and I was thinking,

we know how far we are from terminators coming and

that seemed to be a little bit far-fetched concern.

But I was living my work life

with a real concern I felt no one was talking about,

which was the lack of representation in AI.

At that time, I guess after Daphne has left,

I was the only woman faculty at Stanford AI Lab.

We're having very small,

around 15% of women graduate students,

and we really don't see anybody from

the underrepresented minority groups

in Stanford AI program.

This is a national or even worldwide issue,

so it wasn't just Stanford.

Frankly, it still needs a lot of work today.

Exactly. How do we do this?

Well, I got together with

my former student Ugawa Sakofski,

and also a long-term educator of

STEM topics Doctor Rick Summer

from Stanford Pre-Collegiate Study Program,

and thought about inviting

high schoolers at that time women,

high-school young women to participate

in a summer program to inspire them to learn AI,

and that was how it started in 2015,

and 2017 we got a lot of encouragement

and support from people like

Jensen and Lori Hung and Melinda Gates,

and we formed a national non-profit AI4ALL,

which is really committed to training or

shaping tomorrow's leaders for AI.

From students of all walks of life,

especially the traditionally

under-served and underrepresented communities.

Till today, we've had

many summer camps and summer programs

across the country

more than 15 universities are involved,

and we have online curriculum

to encourage students as well as college

pathway programs to continue support

these students' career by

matching them with internships and mentors.

It's a continuous effort of

encouraging students of all walks of life.

I remember back then,

I think your group was printing

these really cool t-shirts that asked the question.

AI will change the world,

who will change AI?

I thought the answer of making sure

everyone can come in and participate,

that was a great answer.

Still an important question today.

That's a great thought and I think that

takes us toward the end of the interview.

Any final thoughts for the people watching this?

Still, that this is a very nascent field.

As you said, Andrew,

we're still in the middle of this.

I still feel there's just so many questions

that I wake up

excited to work on with my students in the lab,

and I think there's

a lot more opportunities for the young people

out there who want to learn and

contribute and shape tomorrow's AI.

Well said [inaudible] that's very inspiring,

really great to chat with you,

and thank you for attending this video.

Thank you. It's fun to have these conversations.