All language subtitles for 1. What is Deep Learning

What Alison should know--
1

What is the Internet, anyway?
2

Internet is that massive computer network,
3

the one that's becoming really big now.
4

What do you mean that's big?
5

How does (mumbles), you write to it, like mail?
6

No, a lot of people use it to communicate with,
7

I guess they can communicate with NBC,
8

writers and producers.
9

Alison, can you explain what Internet is?
10

(dramatic music)
11

How amazing is that?
12

Just over 20 years ago,
13

people didn't even know what the Internet was
14

and today we can't even imagine our lives without it.
15

Welcome to the Deep Learning A-Z course.
16

My name is Kirill Eremenko,
17

and along with the co-instructor Hadelin de Ponteves,
18

we're super excited to have you on board.
19

And today we're going to give you a quick overview
20

of what deep learning is and why it's picking up right now.
21

So, let's get started.
22

Why did we have a look at that clip?
23

And what is this photo over here?
24

Well, that clip was from 1994,
25

this is a photo of a computer from 1980.
26

And the reason why we're kinda delving into history
27

a little bit is because neural networks,
28

along with deep learning,
29

have been around for quite some time,
30

and they've only started picking up now
31

and impacting the world right now.
32

But if you look back at the '80s you'll see that,
33

even though they were invented in the '60s and '70s,
34

they really caught onto a trend or caught wind in the '80s.
35

People started talking about them a lot,
36

there was a lot of research in that area,
37

and everybody thought that deep learning or neural networks
38

were this new thing that is going to impact the world,
39

is going to change everything,
40

is gonna solve all the world problems,
41

and then it kind of slowly died off over the next decade.
42

So what happened?
43

Why did the neural networks not survive
44

and not change the world?
45

The reason for that, that they were just not good enough,
46

that they're not that good at predicting things
47

and not that good at modeling
48

and, basically, just not a good invention,
49

or is there another reason?
50

Well, actually, there is another reason.
51

And the reason is in front of us,
52

it's the fact that technology back then
53

was not up to the right standard
54

to facilitate neural networks.
55

In order for neural networks and deep learning
56

to work properly, you need two things.
57

You need data, and you need a lot of data,
58

and you need processing power,
59

you need strong computers to process that data
60

and facilitate the neural networks.
61

So let's have a look at how data,
62

or storage of data, has evolved over the years,
63

and then we'll look at how technology has evolved.
64

So, here we've got three years.
65

1956, 1980, 2017.
66

How did storage look back in 1956?
67

Well, there is a hard drive,
68

and that hard drive is only a 5, wait for it,
69

megabyte hard drive.
70

That's 5 megabytes right there on the forklift,
71

the size of a small room, that's a hard drive
72

being transported to another location on a plane.
73

That is what storage looked like in 1956.
74

You had to pay, a company had to pay $2,500,
75

of those days dollars, to rent that hard drive.
76

To rent it, not buy it, to rent it for one month.
77

In 1980, the situation improved a little bit.
78

So here we got a 10 megabytes hard drive for $3,500.
79

It's still very expensive and only 10 megabytes,
80

so that's like one photo these days.
81

And today, in 2017, we've got a 256 gigabyte SSD card
82

for $150 which can fit on your finger.
83

And if you're watching this video,
84

a year later or, like, in 2019 or 2025,
85

you're probably laughing to yourself
86

because by then you have even stronger storage capacity.
87

But, nevertheless, the point stands.
88

If we compare these across the board
89

and, without even taking price and size into consideration,
90

just the capacity of whatever was trending at the time,
91

so from 1956 to 1980 capacity increased about double
92

and then it increased about 25,600 times.
93

And the length of the period is not that different,
94

from 1956 to 1980 is 24 years,
95

from 1980 to 2017, 37 years,
96

so not that much of an increase in time
97

but a huge jump in technological progress,
98

and that stands to show that this is not a linear trend,
99

this is an exponential growth in technology.
100

And if we take into account price and size,
101

it'll be in the millions of increase.
102

And here we actually have a chart on a logarithmic scale,
103

so if we plot the hard drive cost per gigabyte
104

you'll see that it looks something like this.
105

We're very quickly approaching zero.
106

Right now you can get storage on DropBox and Google Drive,
107

which doesn't cost you anything, Cloud storage,
108

and that's going to continue.
109

And, in fact, over the years,
110

this is going to go even further.
111

Right now, scientists are even looking into
112

using DNA for storage.
113

Right now it's quite expensive,
114

it costs $7,000 to synthesize 2 megabytes of data
115

and then another $2,000 to read it,
116

but that kinda reminds you of this whole situation
117

of the hard drive and the plane,
118

that this is gonna be mitigated very, very quickly
119

with this exponential growth.
120

10 years from now, 20 years from now,
121

everybody's gonna be using DNA storage
122

if we go down this direction.
123

Here are some stats on all that.
124

You can explore this further, maybe pause the video
125

if you want to read a bit more about this.
126

This is from nature.com.
127

And, basically, you can store all of the world's data
128

in just 1 kilogram of DNA storage.
129

Or you can store about one billion terabytes of data
130

in one gram of DNA storage.
131

So, that's just something to show
132

how quickly we're progressing
133

and that this is why deep learning is picking up
134

now that we are finally at the stage
135

where we have enough data to train
136

super cool, super sophisticated models.
137

Back then, in the '80s when it was first initiated,
138

it just wasn't the case.
139

And the second thing we talked about is processing capacity.
140

So here we've got an exponential curve,
141

again, on a log scale.
142

It's not ideally portrayed here
143

but on the right you can see it's a log scale.
144

And this is how computers have been evolving.
145

So, again, feel free to pause this slide.
146

This is called Moore's law, you've probably heard of it,
147

how quickly the processing capacity of computers
148

has been evolving.
149

Right now we're somewhere over here,
150

where an average computer you can buy for 1,000 bucks
151

thinks at the speed of the brain of a rat.
152

And by 2025 it'll be the speed of a human, or 2023.
153

And then by 2050 or 2045,
154

we'll surpass all of the humans combined.
155

So, basically, we are entering the era of computers
156

that are extremely powerful,
157

that can process things way faster than we can imagine,
158

and that is what is facilitating deep learning.
159

So, all this brings us to the question,
160

what is deep learning?
161

What is this whole neural network situation?
162

What is going on, what are we even talking about here?
163

And you've probably seen a picture of something like this,
164

so let's dive into it.
165

What is deep learning?
166

This gentleman over here, Geoffrey Hinton,
167

is known as the godfather of deep learning.
168

He did research on deep learning in the '80s
169

and he's done lots and lots of work,
170

lots of research papers he's published in deep learning.
171

Right now he works at Google.
172

So, a lot of the things that we're gonna be talking about
173

actually come from Geoffrey Hinton.
174

You can see he's got a quite a few Youtube videos,
175

he explains things really well, so,
176

highly recommend checking them out.
177

So, the idea behind deep learning is to
178

look at the human brain,
179

and there's gonna be quite a bit of neuroscience coming up
180

in these tutorials,
181

and what we're trying to do here
182

is to mimic how the human brain operates.
183

We don't know that much,
184

we don't know everything about the human brain,
185

but that little amount that we know,
186

we want to mimic it and recreate it.
187

And why is that?
188

Well, because the human brain seems to be
189

one of the most powerful tools on this planet for learning,
190

for learning adapting skills and then applying them.
191

If computers could copy that, then we could just leverage
192

what natural selection has already decided for us,
193

all of those algorithms that it has decided are the best,
194

we're just gonna leverage that, (mumbles).
195

So, let's see how this works.
196

Here we've got some neurons.
197

These are neurons which have been smeared onto glass
198

and then have been looked at under a microscope
199

with some coloring,
200

and you can see what they look like.
201

They have like a body, they have these branches,
202

and they have like tails, and so on,
203

you can see then they have a nucleus inside in the middle.
204

That's basically what a neuron looks like.
205

In the human brain there is approximately
206

a 100 billion neurons altogether.
207

These are individual neurons,
208

these are actually motor neurons
209

because they're bigger, they're easier to see.
210

But nevertheless there's a 100 billion neurons
211

in the human brain,
212

and each neuron is connected
213

to as many as about a 1,000 of its neighbors.
214

So, to give you a picture, this is what it looks like.
215

This is an actual section of the human brain,
216

this is the cerebellum, which is this part of your brain.
217

At the back.
218

It's responsible for
219

motorics and for keeping your balance
220

and some language capabilities and stuff like that.
221

This is just to show how vast, how many neurons there are.
222

There are billions and billions and billions of neurons
223

all connecting your brain.
224

It's not like we're talking about five or 500 or a 1,000
225

or a million, there's billions of neurons in there.
226

Yeah, so, that's what we're gonna be trying to recreate.
227

So, how do we recreate this in a computer?
228

Well, we create an artificial structure,
229

called an artificial neural net,
230

where we have nodes, or neurons.
231

And we're gonna have some neurons for input value,
232

so these are values that you know about a certain situation.
233

For instance, if you're modeling something,
234

you want to predict something,
235

you're always gonna have some input,
236

something to start your predictions off.
237

That's called the input layer.
238

Then you have the output,
239

so that's a value that you want to predict,
240

whether it's a price,
241

whether it's is somebody going to leave the bank
242

or stay in the bank,
243

is this a fraudulent transaction,
244

is this a real transaction, and so on.
245

So that's gonna be output layer.
246

And in between, we're going to have a hidden layer.
247

As you can see, in your brain you have so many neurons,
248

so some information is coming in
249

through your eyes, ears, nose, basically your senses,
250

and then it's not just going right away to the output
251

where you have the result,
252

it's going through all of these billions and billions
253

and billions of neurons before it gets to the output.
254

And this is the whole concept behind it
255

that we're going to model the brain,
256

so we need these hidden layers
257

that are there before the output.
258

So, the input layers neurons are connected
259

to the hidden layer neurons,
260

the hidden layer neurons are connected to output value.
261

So this is pretty cool but what is this all about?
262

Where is the deep learning here?
263

Why is it called deep learning?
264

There's nothing deep in here.
265

Well, this is kind of like an option
266

which one might call shallow learning,
267

where there isn't much indeed going on.
268

But, why is it called deep learning?
269

Well, because then we take this to the next level.
270

We separate it even further
271

and we have not just one hidden layer,
272

we have lots and lots and lots of hidden layers,
273

and then we connect everything,
274

just like in the human brain.
275

Connect everything, interconnect everything,
276

and that's how the input values are processed
277

through all these hidden layers,
278

just like in the human brain, then we have an output value.
279

And now we're talking deep learning.
280

So that's what deep learning is all about
281

on a very abstract level.
282

In the further tutorials we're going to dissect
283

and dive deep into deep learning,
284

and by the end of it you will know
285

what deep learning is all about
286

and you'll know how to apply it in your projects.
287

Super excited about this, can't wait to get started,
288

and I look forward to seeing you on the next tutorial.
289

Until then, enjoy deep learning.