Would you like to inspect the original subtitles? These are the user uploaded subtitles that are being translated:
1
1
00:00:00,190 --> 00:00:01,400
Hello and Welcome back to the course
2
2
00:00:01,400 --> 00:00:02,320
on Deep Learning.
3
3
00:00:02,320 --> 00:00:03,850
Today we're talking about the neuron,
4
4
00:00:03,850 --> 00:00:06,180
which is the basic building block
5
5
00:00:06,180 --> 00:00:07,880
of artificial neural networks.
6
6
00:00:07,880 --> 00:00:09,280
So let's get started.
7
7
00:00:09,280 --> 00:00:11,220
Previously we saw an image which looked like this.
8
8
00:00:11,220 --> 00:00:13,353
And these are actual, real life neurons
9
9
00:00:13,353 --> 00:00:17,820
which have smeared on to glass, colored a little bit,
10
10
00:00:17,820 --> 00:00:19,880
and they are observed through a microscope.
11
11
00:00:19,880 --> 00:00:21,734
So this is what they look like as you can see,
12
12
00:00:21,734 --> 00:00:23,570
quite an interesting structure.
13
13
00:00:23,570 --> 00:00:27,690
A body, and a lot of different tails,
14
14
00:00:27,690 --> 00:00:30,200
kind of branches coming out of them.
15
15
00:00:30,200 --> 00:00:33,420
And this is very interesting but the question is
16
16
00:00:33,420 --> 00:00:36,040
how can we recreate that in a machine?
17
17
00:00:36,040 --> 00:00:38,900
Because we really need to recreate that in a machine
18
18
00:00:38,900 --> 00:00:41,960
since the whole purpose of Deep Learning is to
19
19
00:00:41,960 --> 00:00:44,523
mimic how the human brain works.
20
20
00:00:46,030 --> 00:00:48,560
In the hopes that by doing so,
21
21
00:00:48,560 --> 00:00:50,890
we are going to create something amazing.
22
22
00:00:50,890 --> 00:00:53,110
We are going to create an amazing infrastructure
23
23
00:00:53,110 --> 00:00:55,110
for machines to be able to learn.
24
24
00:00:55,110 --> 00:00:56,740
And why do we hope for that?
25
25
00:00:56,740 --> 00:00:59,200
Well because the human brain is,
26
26
00:00:59,200 --> 00:01:01,660
well just happens to be one of the most powerful
27
27
00:01:01,660 --> 00:01:04,490
learning tools on the planet,
28
28
00:01:04,490 --> 00:01:07,220
or like learning mechanisms on the planet.
29
29
00:01:07,220 --> 00:01:09,710
And we just hope that if we recreate that
30
30
00:01:09,710 --> 00:01:11,250
we'll have something as awesome as that.
31
31
00:01:11,250 --> 00:01:12,870
So our challenge right now,
32
32
00:01:12,870 --> 00:01:14,410
our very first step to creating
33
33
00:01:14,410 --> 00:01:16,100
artificial neural networks,
34
34
00:01:16,100 --> 00:01:18,270
is to recreate a neuron.
35
35
00:01:18,270 --> 00:01:19,103
So how do we do that?
36
36
00:01:19,103 --> 00:01:21,300
Well, first let's take a closer look
37
37
00:01:21,300 --> 00:01:23,187
at what it actually is.
38
38
00:01:23,187 --> 00:01:26,070
This image was first created by
39
39
00:01:26,070 --> 00:01:27,100
a Spanish neural scientist,
40
40
00:01:27,100 --> 00:01:31,450
Santiago Ramón y Cajal, in 1899.
41
41
00:01:33,080 --> 00:01:36,900
And what he did was he dyed neurons in
42
42
00:01:36,900 --> 00:01:37,930
actual brain tissue and looked
43
43
00:01:37,930 --> 00:01:39,770
at them under a microscope.
44
44
00:01:39,770 --> 00:01:41,020
And while he was looking at them
45
45
00:01:41,020 --> 00:01:42,430
he actually drew what he saw.
46
46
00:01:42,430 --> 00:01:43,440
And this is what he saw.
47
47
00:01:43,440 --> 00:01:45,570
He saw two neurons or two large neurons
48
48
00:01:45,570 --> 00:01:46,830
over there at the top,
49
49
00:01:46,830 --> 00:01:49,950
which had all these branches coming out of them
50
50
00:01:49,950 --> 00:01:53,960
towards their top parts and then each had a
51
51
00:01:53,960 --> 00:01:58,960
rod or thread coming out towards the bottom, very long one.
52
52
00:01:59,400 --> 00:02:01,550
And that's what he saw.
53
53
00:02:01,550 --> 00:02:03,560
And now, you know, technology has advanced
54
54
00:02:03,560 --> 00:02:07,020
quite a lot and we have seen neurons much closer
55
55
00:02:07,020 --> 00:02:08,969
and more detailed and now we can actually draw
56
56
00:02:08,969 --> 00:02:11,940
what it looks like diagrammatically.
57
57
00:02:11,940 --> 00:02:13,290
So let's have a look at that.
58
58
00:02:13,290 --> 00:02:15,300
Here's a neuron, this is what it looks like.
59
59
00:02:15,300 --> 00:02:20,300
Very similar to what Santiago Ramón drew over here.
60
60
00:02:21,000 --> 00:02:23,010
Here in this neuron what we can see is that
61
61
00:02:23,010 --> 00:02:26,420 line:15%
its got a body, that's the main part of the neuron.
62
62
00:02:26,420 --> 00:02:28,050
And then its got some branches at the top,
63
63
00:02:28,050 --> 00:02:29,050
which are called dendrites.
64
64
00:02:29,050 --> 00:02:30,540
And its also got an axon,
65
65
00:02:30,540 --> 00:02:33,380
which is that long tail of the neuron.
66
66
00:02:33,380 --> 00:02:36,620
So what are these dendrites for and what's the axon for.
67
67
00:02:36,620 --> 00:02:39,860
Well, the key point to understand here is that
68
68
00:02:39,860 --> 00:02:44,250
neurons by themselves are pretty much useless.
69
69
00:02:44,250 --> 00:02:45,900
It's like an ant.
70
70
00:02:45,900 --> 00:02:47,970
An ant on its own can't do much,
71
71
00:02:47,970 --> 00:02:51,110
like 5 ants together maybe they can pick something up.
72
72
00:02:51,110 --> 00:02:54,040
But again, they can't build an ant hill,
73
73
00:02:54,040 --> 00:02:55,360
they can't establish a colony,
74
74
00:02:55,360 --> 00:02:59,290
they can't work together as a huge organism.
75
75
00:02:59,290 --> 00:03:01,460
But at the same time, when you have lots and lots of ants,
76
76
00:03:01,460 --> 00:03:04,220
like you have a million ants, they can build a whole colony,
77
77
00:03:04,220 --> 00:03:05,680
they can build an ant hill.
78
78
00:03:05,680 --> 00:03:06,520
Same thing with neurons.
79
79
00:03:06,520 --> 00:03:07,740
By itself it's not that strong,
80
80
00:03:07,740 --> 00:03:09,750
but when you have lots of neurons together,
81
81
00:03:09,750 --> 00:03:12,018
they work together to do magic.
82
82
00:03:12,018 --> 00:03:14,330
And how do they work together? That's a question.
83
83
00:03:14,330 --> 00:03:16,610
Well, that's what the dendrites and axon are for.
84
84
00:03:16,610 --> 00:03:18,370
So the dendrites are kind of like the
85
85
00:03:18,370 --> 00:03:19,960
receivers of the signal for the neuron,
86
86
00:03:19,960 --> 00:03:23,100
and axon is the transmitter of the signal for the neuron.
87
87
00:03:23,100 --> 00:03:26,450
And here's an image of how it all works conceptually.
88
88
00:03:26,450 --> 00:03:27,920
So at the top you got a neuron,
89
89
00:03:27,920 --> 00:03:31,140
and you can see that its dendrites are connected
90
90
00:03:31,140 --> 00:03:33,870
to axons of other neurons that are like
91
91
00:03:33,870 --> 00:03:35,790
even further away above it.
92
92
00:03:35,790 --> 00:03:38,880
And then the signal from this neuron travels down
93
93
00:03:38,880 --> 00:03:41,726
its axon and connects or passes onto
94
94
00:03:41,726 --> 00:03:43,530
the dendrites of the other neuron.
95
95
00:03:43,530 --> 00:03:44,930
And that's how they're connected.
96
96
00:03:44,930 --> 00:03:46,670
And in that small image over there,
97
97
00:03:46,670 --> 00:03:47,730
you can see that
98
98
00:03:47,730 --> 00:03:51,893
the axon doesn't actually touch the dendrite.
99
99
00:03:52,764 --> 00:03:54,810
(laughs) A lot of machine learning,
100
100
00:03:54,810 --> 00:03:56,674
or a few machine learning scientists
101
101
00:03:56,674 --> 00:03:59,100
are very adamant about the fact
102
102
00:03:59,100 --> 00:04:00,200
that it doesn't touch.
103
103
00:04:02,690 --> 00:04:04,680
It doesn't touch, it has been proven that
104
104
00:04:04,680 --> 00:04:06,870
there is no physical connection there.
105
105
00:04:06,870 --> 00:04:09,080
But the point that we are interested in
106
106
00:04:09,080 --> 00:04:11,200
is that that connection between them,
107
107
00:04:11,200 --> 00:04:15,160
that the whole concept of the signal being passed,
108
108
00:04:15,160 --> 00:04:16,220
that's called the synapse.
109
109
00:04:16,220 --> 00:04:18,470 line:15%
You can see over there, in that little image,
110
110
00:04:20,041 --> 00:04:22,610 line:15%
that figure bracket is synapse.
111
111
00:04:22,610 --> 00:04:23,880
That's the term we're going to be using.
112
112
00:04:23,880 --> 00:04:27,830
Instead of calling our artificial neurons,
113
113
00:04:27,830 --> 00:04:28,950
the lines we're gonna have,
114
114
00:04:28,950 --> 00:04:30,580
or the connectors for artificial neurons
115
115
00:04:30,580 --> 00:04:32,517
we're not be calling them axons or dendrites,
116
116
00:04:32,517 --> 00:04:34,110
because then the question is
117
117
00:04:34,110 --> 00:04:35,120
whose connection is this?
118
118
00:04:35,120 --> 00:04:36,070
Is it that neuron's or is it this neuron's?
119
119
00:04:36,070 --> 00:04:39,055
We're just going to call them synapses.
120
120
00:04:39,055 --> 00:04:42,721
And that kind of just answers all the questions.
121
121
00:04:42,721 --> 00:04:45,050
I mean it's basically just where the signal is passed.
122
122
00:04:45,050 --> 00:04:47,097
Doesn't matter who that element belongs to.
123
123
00:04:47,097 --> 00:04:50,005
That's just a representation of the signal
124
124
00:04:50,005 --> 00:04:51,870
being passed and we see that just now.
125
125
00:04:51,870 --> 00:04:54,773
So basically that's how a neuron works.
126
126
00:04:56,340 --> 00:04:59,653
Let's move on to how we're going to represent
127
127
00:04:59,653 --> 00:05:02,503
neurons or how we're going to create neurons in machines.
128
128
00:05:03,665 --> 00:05:06,330
So now we're moving away from neural science
129
129
00:05:06,330 --> 00:05:09,280
and moving into technology.
130
130
00:05:09,280 --> 00:05:10,250
And here we go.
131
131
00:05:10,250 --> 00:05:11,590
So, here's our neuron,
132
132
00:05:11,590 --> 00:05:13,640
also sometimes called the node.
133
133
00:05:13,640 --> 00:05:15,938
The neuron gets some input signals.
134
134
00:05:15,938 --> 00:05:18,290
And it has an output signal.
135
135
00:05:18,290 --> 00:05:20,930
So dendrites and axons, remember?
136
136
00:05:20,930 --> 00:05:23,083
But again, we're gonna call these synopses.
137
137
00:05:25,860 --> 00:05:26,693
These input signals,
138
138
00:05:26,693 --> 00:05:27,900
we're going to represent them with
139
139
00:05:27,900 --> 00:05:28,980
other neurons as well.
140
140
00:05:28,980 --> 00:05:30,690
So, in this specific case,
141
141
00:05:30,690 --> 00:05:31,523
you can see that
142
142
00:05:31,523 --> 00:05:33,760
this neuron, this green neuron,
143
143
00:05:33,760 --> 00:05:35,780
is getting signals from yellow neurons.
144
144
00:05:35,780 --> 00:05:37,410
And in this course, we are going to try
145
145
00:05:37,410 --> 00:05:40,530
to stick to a certain color coding regime,
146
146
00:05:40,530 --> 00:05:42,470
where yellow means an input layer.
147
147
00:05:42,470 --> 00:05:45,540
So basically all the neurons that are
148
148
00:05:45,540 --> 00:05:48,780
on the outer layer, on the first front of
149
149
00:05:49,750 --> 00:05:51,540
where the signals coming in.
150
150
00:05:51,540 --> 00:05:54,990
By signal, it might be a bit of an overkill
151
151
00:05:56,030 --> 00:05:57,040
to call this a signal.
152
152
00:05:57,040 --> 00:05:58,680
It's just basically input value.
153
153
00:05:58,680 --> 00:06:01,180
So you know how even like in a simple
154
154
00:06:01,180 --> 00:06:03,211
linear regression you have input values,
155
155
00:06:03,211 --> 00:06:04,950
and then you have a predicted value.
156
156
00:06:04,950 --> 00:06:05,783
Same thing here.
157
157
00:06:05,783 --> 00:06:07,120
So you have input values,
158
158
00:06:07,120 --> 00:06:08,940
and there they are, the yellow ones.
159
159
00:06:08,940 --> 00:06:10,630
And on the right to you we see just now
160
160
00:06:10,630 --> 00:06:12,578
it'll be red, it'll be the output value.
161
161
00:06:12,578 --> 00:06:15,620
The thing that I wanted to point out here is that
162
162
00:06:15,620 --> 00:06:17,010
in this specific example we are looking at
163
163
00:06:17,010 --> 00:06:19,660
a neuron which is getting its signals from
164
164
00:06:19,660 --> 00:06:21,220
the input layer neurons.
165
165
00:06:21,220 --> 00:06:22,460
So they are also neurons but
166
166
00:06:22,460 --> 00:06:23,920
they are input layer neurons.
167
167
00:06:23,920 --> 00:06:26,590
Sometimes you'll have neurons which
168
168
00:06:26,590 --> 00:06:30,420
get their signal from other hidden layer neurons,
169
169
00:06:30,420 --> 00:06:31,670
so from other green neurons.
170
170
00:06:31,670 --> 00:06:32,933
And the concept is gonna be exactly the same.
171
171
00:06:32,933 --> 00:06:35,950
Just in this case, for simplicity's sake,
172
172
00:06:35,950 --> 00:06:37,510
we're portraying this example.
173
173
00:06:37,510 --> 00:06:38,960
And in terms of the input layer,
174
174
00:06:38,960 --> 00:06:40,360
the way to think about it is
175
175
00:06:42,990 --> 00:06:44,857
in the analogy of the human brain,
176
176
00:06:44,857 --> 00:06:48,030
the input layer is your senses, right.
177
177
00:06:48,030 --> 00:06:49,810
So whatever you can see, hear,
178
178
00:06:49,810 --> 00:06:52,380
feel, touch or smell.
179
179
00:06:52,380 --> 00:06:53,340
And of course,
180
180
00:06:53,340 --> 00:06:55,760
there's a lot of things you can see,
181
181
00:06:55,760 --> 00:06:57,670
there's a lot of information coming in.
182
182
00:06:57,670 --> 00:06:58,780
But those are your...
183
183
00:06:58,780 --> 00:07:00,040
that's what your brain is limited to,
184
184
00:07:00,040 --> 00:07:02,880
it's pretty much a (laughs)
185
185
00:07:02,880 --> 00:07:04,770
it's pretty much lives in a box
186
186
00:07:04,770 --> 00:07:07,690
made out of bones and it's only...
187
187
00:07:07,690 --> 00:07:09,370
It's a mind blowing fact to think about.
188
188
00:07:09,370 --> 00:07:13,080
Your brain is just locked in a black box,
189
189
00:07:13,080 --> 00:07:13,913
and the only thing...
190
190
00:07:13,913 --> 00:07:15,350
and it can't see, it can't hear,
191
191
00:07:15,350 --> 00:07:16,210
the only thing it's getting
192
192
00:07:16,210 --> 00:07:18,150
is electrical impulses coming from
193
193
00:07:18,150 --> 00:07:20,410
these organs that you have,
194
194
00:07:20,410 --> 00:07:22,663
which are called your ears, nose, eyes,
195
195
00:07:23,520 --> 00:07:28,520
your sense of touch and whatever... and your taste.
196
196
00:07:28,590 --> 00:07:30,120
It's just getting signals but
197
197
00:07:30,120 --> 00:07:32,090
it basically lives in this dark black box
198
198
00:07:32,090 --> 00:07:35,960
and it's making sense of the world through your senses.
199
199
00:07:35,960 --> 00:07:37,413
It's phenomenal.
200
200
00:07:38,910 --> 00:07:41,537
So you have these inputs that are coming in,
201
201
00:07:41,537 --> 00:07:44,192
and in terms of human brain those are your five senses,
202
202
00:07:44,192 --> 00:07:47,540
in terms of machine learning or deep learning,
203
203
00:07:47,540 --> 00:07:50,530
that is basically your input values,
204
204
00:07:50,530 --> 00:07:52,066
so your independent variables,
205
205
00:07:52,066 --> 00:07:52,899
and we will get to that in a second.
206
206
00:07:52,899 --> 00:07:54,923
So your input values,
207
207
00:07:56,310 --> 00:07:58,780
the signal is passed through synapses to your neuron,
208
208
00:07:58,780 --> 00:08:00,700
and then your neuron has an output value,
209
209
00:08:00,700 --> 00:08:03,440
that it passes further on down the chain.
210
210
00:08:03,440 --> 00:08:05,160
In this specific case, in terms of color coding,
211
211
00:08:05,160 --> 00:08:06,940
again yellow means input layer.
212
212
00:08:06,940 --> 00:08:08,520
So we kind of simplifying everything here.
213
213
00:08:08,520 --> 00:08:11,050
We're saying we're only gonna have like the input layer,
214
214
00:08:11,050 --> 00:08:13,150
then we're gonna have one hidden layer,
215
215
00:08:13,150 --> 00:08:15,010
with the green, which is a hidden layer,
216
216
00:08:15,010 --> 00:08:17,410
and then we're gonna have our output layer right away.
217
217
00:08:17,410 --> 00:08:20,253
So just so that we can get used to those colors for now.
218
218
00:08:21,470 --> 00:08:23,950
So there we go, that's the basic structure.
219
219
00:08:23,950 --> 00:08:26,020
So now let's look at a bit more detail
220
220
00:08:26,020 --> 00:08:28,310
at these different elements that we have.
221
221
00:08:28,310 --> 00:08:29,870
So we got the input layer.
222
222
00:08:29,870 --> 00:08:31,000
And what do we have here?
223
223
00:08:31,000 --> 00:08:33,840
Well, we have these inputs which are
224
224
00:08:33,840 --> 00:08:35,410
in fact independent variables.
225
225
00:08:35,410 --> 00:08:36,360
So independent variable one,
226
226
00:08:36,360 --> 00:08:37,193
independent variable two,
227
227
00:08:37,193 --> 00:08:38,570
and independent variable m.
228
228
00:08:38,570 --> 00:08:39,880
The important thing to remember here,
229
229
00:08:39,880 --> 00:08:42,700
is that these independent variables
230
230
00:08:42,700 --> 00:08:44,650
are all for one single observation.
231
231
00:08:44,650 --> 00:08:47,530
So think of it as one row in your data base.
232
232
00:08:47,530 --> 00:08:48,930
One observation.
233
233
00:08:48,930 --> 00:08:51,303
You just take all of the independent variables,
234
234
00:08:52,158 --> 00:08:54,841
maybe it's the age of the person,
235
235
00:08:54,841 --> 00:08:57,727
the amount of money in their bank account,
236
236
00:08:57,727 --> 00:09:00,690
how do they drive or walk to work,
237
237
00:09:00,690 --> 00:09:02,835
what method of transportation do they use.
238
238
00:09:02,835 --> 00:09:06,030
But that's all descriptions of one specific person,
239
239
00:09:06,030 --> 00:09:08,970
that you are, either you're training your model on,
240
240
00:09:08,970 --> 00:09:11,724
or you're performing some prediction on.
241
241
00:09:11,724 --> 00:09:14,010
And the other thing you need to know
242
242
00:09:14,010 --> 00:09:15,050
about these variables is that
243
243
00:09:15,050 --> 00:09:16,700
you need to standardize them.
244
244
00:09:16,700 --> 00:09:19,140
You need to either standardize them which means
245
245
00:09:19,140 --> 00:09:21,250
make sure they have a mean of zero and variance one,
246
246
00:09:21,250 --> 00:09:23,259
or you can also sometimes and
247
247
00:09:23,259 --> 00:09:25,840
Hadelin will point out these tricks
248
248
00:09:25,840 --> 00:09:27,960
in a bit more detail,
249
249
00:09:27,960 --> 00:09:29,730
perhaps in the practical tutorials
250
250
00:09:29,730 --> 00:09:30,890
you might come across these,
251
251
00:09:30,890 --> 00:09:32,880
sometimes you might want to not standardize
252
252
00:09:32,880 --> 00:09:34,900
you might wanna normalize them.
253
253
00:09:34,900 --> 00:09:37,264
Meaning that instead of making sure that
254
254
00:09:37,264 --> 00:09:38,900
mean is zero and variance is one,
255
255
00:09:38,900 --> 00:09:42,510
you just subtract the minimum value and
256
256
00:09:42,510 --> 00:09:44,450
then you divide it by maximum minus minimum,
257
257
00:09:44,450 --> 00:09:46,187
so by the range of your values and
258
258
00:09:46,187 --> 00:09:49,067
therefore you get values between zero and one.
259
259
00:09:51,190 --> 00:09:53,030
Depend on the scenario you might wanna do one
260
260
00:09:53,030 --> 00:09:54,250
or the other but basically you want
261
261
00:09:54,250 --> 00:09:56,880
all of these variables to be quite similar,
262
262
00:09:56,880 --> 00:10:01,200
in about the same range of values.
263
263
00:10:01,200 --> 00:10:02,233
Why's that?
264
264
00:10:03,483 --> 00:10:04,316
Well all of these values are going to
265
265
00:10:04,316 --> 00:10:05,640
go into a neural network where as
266
266
00:10:05,640 --> 00:10:07,842
we all see just now they will be added up and
267
267
00:10:07,842 --> 00:10:10,280
multiplied by weights added up and so on.
268
268
00:10:10,280 --> 00:10:13,020
It's just going to be easier for
269
269
00:10:13,020 --> 00:10:14,763
the neural network to process them
270
270
00:10:14,763 --> 00:10:16,763
if they are all about the same.
271
271
00:10:19,271 --> 00:10:21,729
And that's just how it is going to
272
272
00:10:21,729 --> 00:10:24,120
be able to work properly.
273
273
00:10:24,120 --> 00:10:26,280
And if you want to read more about
274
274
00:10:26,280 --> 00:10:28,870
standardization, normalization and other things
275
275
00:10:28,870 --> 00:10:30,560
you can do with your input variables,
276
276
00:10:30,560 --> 00:10:34,650
a good additional reading paper is called
277
277
00:10:34,650 --> 00:10:38,450
Efficient BackProp by Yan LeCun 1998,
278
278
00:10:38,450 --> 00:10:40,297 line:15%
the link's over there.
279
279
00:10:40,297 --> 00:10:41,430
So Yan LeCun, we're actually going to
280
280
00:10:41,430 --> 00:10:44,720
talk about this phenomenal person
281
281
00:10:44,720 --> 00:10:45,850
in the place of Deep Learning
282
282
00:10:45,850 --> 00:10:48,240
in the part of the course where
283
283
00:10:48,240 --> 00:10:50,616
we're talking about illusional neural networks.
284
284
00:10:50,616 --> 00:10:53,180
You'll see that this is definitely a person
285
285
00:10:53,180 --> 00:10:55,180
who knows what he's talking about.
286
286
00:10:55,180 --> 00:10:57,630
He's a close friend of Geoffrey Hinton,
287
287
00:10:57,630 --> 00:11:00,790
who we already seen, who we've already mentioned.
288
288
00:11:00,790 --> 00:11:04,040
So in this paper you will learn more about
289
289
00:11:04,040 --> 00:11:05,590
standardization and normalization.
290
290
00:11:05,590 --> 00:11:07,070
But you can pick up lots of
291
291
00:11:07,070 --> 00:11:09,000
other different tips and tricks and
292
292
00:11:09,000 --> 00:11:11,180
be a good source of additional reading
293
293
00:11:11,180 --> 00:11:12,490
as you go through this course.
294
294
00:11:12,490 --> 00:11:14,010
So check it out if you're interested
295
295
00:11:14,010 --> 00:11:16,190
in some additional reading.
296
296
00:11:16,190 --> 00:11:18,590
There we go, so that's what we need to do
297
297
00:11:18,590 --> 00:11:20,280
with the variables.
298
298
00:11:20,280 --> 00:11:23,090
And here we've got the output value.
299
299
00:11:23,090 --> 00:11:25,010
So what can our output value be?
300
300
00:11:25,010 --> 00:11:26,220
Well we've got a couple of options.
301
301
00:11:26,220 --> 00:11:27,950
Output value can be,
302
302
00:11:27,950 --> 00:11:30,040
it can be continuous, for instance, price;
303
303
00:11:30,040 --> 00:11:31,550
it can be binary, for instance,
304
304
00:11:31,550 --> 00:11:33,610
a person will exit or stay;
305
305
00:11:33,610 --> 00:11:35,913
or it can be a categorical variable.
306
306
00:11:37,160 --> 00:11:39,180
If it's a categorical variable,
307
307
00:11:39,180 --> 00:11:41,010
the important thing to remember here is that
308
308
00:11:41,010 --> 00:11:43,790
in that case, your output value won't be just one,
309
309
00:11:43,790 --> 00:11:45,920
it'll be several output values,
310
310
00:11:45,920 --> 00:11:47,920
because these will be your dummy variables,
311
311
00:11:47,920 --> 00:11:51,260
which will be representing your categories.
312
312
00:11:51,260 --> 00:11:53,193
And that's just how it works.
313
313
00:11:54,110 --> 00:11:55,400
Just important to remember that,
314
314
00:11:55,400 --> 00:11:58,320
in that case that's how you're going to be getting
315
315
00:11:58,320 --> 00:12:02,243
your categories out of the artificial neural network.
316
316
00:12:02,243 --> 00:12:04,170
But let's go back to our simple case
317
317
00:12:04,170 --> 00:12:05,640
of one output value.
318
318
00:12:05,640 --> 00:12:10,080
And now one more point, a point I've already made,
319
319
00:12:10,080 --> 00:12:12,533
I just want to reiterate this point.
320
320
00:12:12,533 --> 00:12:15,360
On the left you've got a single observation,
321
321
00:12:15,360 --> 00:12:17,630
so one row from your data set,
322
322
00:12:17,630 --> 00:12:19,850
and on the right you have a single observation as well.
323
323
00:12:19,850 --> 00:12:22,080
That is the same observation.
324
324
00:12:22,080 --> 00:12:25,420
So important to remember that whatever inputs
325
325
00:12:25,420 --> 00:12:27,370
you're putting in, that's for one row,
326
326
00:12:27,370 --> 00:12:28,520
and then the output you get back is
327
327
00:12:28,520 --> 00:12:30,000
for that exact same row.
328
328
00:12:30,000 --> 00:12:32,490
Or if you're training your neural network then
329
329
00:12:32,490 --> 00:12:34,380
you're putting the inputs in for that one row,
330
330
00:12:34,380 --> 00:12:36,680
you're putting the output in for that one row.
331
331
00:12:37,967 --> 00:12:39,250
So if you wanna simplify the complexity,
332
332
00:12:39,250 --> 00:12:42,170
think of it as like a simple linear regression,
333
333
00:12:42,170 --> 00:12:43,900
or a multi-variant linear regression.
334
334
00:12:43,900 --> 00:12:46,270
So you're putting in your values,
335
335
00:12:46,270 --> 00:12:47,583
you have your output.
336
336
00:12:47,583 --> 00:12:49,610
There's no question about it
337
337
00:12:49,610 --> 00:12:51,660
when we are talking about things like regression,
338
338
00:12:51,660 --> 00:12:52,700
because we're so used to it.
339
339
00:12:52,700 --> 00:12:54,890
Same thing here. It's nothing too complex.
340
340
00:12:54,890 --> 00:12:56,030
We're just putting in values,
341
341
00:12:56,030 --> 00:12:56,863
we're getting an output.
342
342
00:12:56,863 --> 00:12:58,270
But just remember that every time
343
343
00:12:58,270 --> 00:12:59,500
it's one row that you're dealing with.
344
344
00:12:59,500 --> 00:13:01,730
So you don't get confused and start putting in
345
345
00:13:01,730 --> 00:13:05,400
like thinking these are different rows that
346
346
00:13:05,400 --> 00:13:07,860
you're putting into your artificial
347
347
00:13:07,860 --> 00:13:09,060
neural network or something.
348
348
00:13:09,060 --> 00:13:11,170
This is all just values in that one row.
349
349
00:13:11,170 --> 00:13:12,240
So different observation,
350
350
00:13:12,240 --> 00:13:13,900
different characteristics of,
351
351
00:13:13,900 --> 00:13:16,730
or attributes relating to that one observation.
352
352
00:13:16,730 --> 00:13:17,630
Every single time.
353
353
00:13:19,070 --> 00:13:20,280
Okay so next thing that we wanna
354
354
00:13:20,280 --> 00:13:24,740
talk about here is the synapses.
355
355
00:13:24,740 --> 00:13:25,890
Here we've got synapses and
356
356
00:13:25,890 --> 00:13:28,840
they all actually get assigned weights.
357
357
00:13:28,840 --> 00:13:31,740
We're gonna talk more about weights further down,
358
358
00:13:31,740 --> 00:13:36,500
but in short, weights are crucial to
359
359
00:13:36,500 --> 00:13:38,560
artificial neural networks functioning.
360
360
00:13:38,560 --> 00:13:41,710
Because weights are how neural networks learn.
361
361
00:13:41,710 --> 00:13:44,320
By adjusting the weights,
362
362
00:13:44,320 --> 00:13:47,340
the neural network decides in every single case,
363
363
00:13:47,340 --> 00:13:49,490
what signal is important and what signal
364
364
00:13:49,490 --> 00:13:51,050
is not important to a certain neuron,
365
365
00:13:51,050 --> 00:13:52,420
what signal gets passed along and
366
366
00:13:52,420 --> 00:13:54,250
what signal doesn't get passed along,
367
367
00:13:54,250 --> 00:13:56,240
or to what strength, to what extent
368
368
00:13:56,240 --> 00:13:57,670
signals get passed along.
369
369
00:13:57,670 --> 00:13:59,240
So weights are crucial,
370
370
00:13:59,240 --> 00:14:02,390
they are the things that get adjusted
371
371
00:14:02,390 --> 00:14:04,100
through the process of learning.
372
372
00:14:04,100 --> 00:14:06,100
When you're training your artificial neural network,
373
373
00:14:06,100 --> 00:14:08,450
you're basically adjusting all of the weights
374
374
00:14:08,450 --> 00:14:09,940
in all of the synapses across this
375
375
00:14:09,940 --> 00:14:11,310
whole neural network and
376
376
00:14:11,310 --> 00:14:13,660
that's where gradient descent and
377
377
00:14:15,010 --> 00:14:16,770
back propagation come into play and
378
378
00:14:16,770 --> 00:14:19,550
those are concepts that we'll also discuss.
379
379
00:14:19,550 --> 00:14:21,310
So basically those are the weights.
380
380
00:14:21,310 --> 00:14:23,710
That's all you need to know for now.
381
381
00:14:23,710 --> 00:14:24,543
Here we've got the neuron.
382
382
00:14:24,543 --> 00:14:26,960
So signals go into the neuron and
383
383
00:14:26,960 --> 00:14:28,340
what happens in the neuron?
384
384
00:14:28,340 --> 00:14:30,690
So this is the interesting part.
385
385
00:14:30,690 --> 00:14:32,150
We're talking about the neuron today,
386
386
00:14:32,150 --> 00:14:33,800
what happens inside the neuron?
387
387
00:14:33,800 --> 00:14:35,450
So, a few things happen.
388
388
00:14:35,450 --> 00:14:37,980
First thing, and the first step is that
389
389
00:14:37,980 --> 00:14:41,260
all of these values that it's getting, get added up.
390
390
00:14:41,260 --> 00:14:45,430
So it takes the added, so the weighted sum
391
391
00:14:45,430 --> 00:14:48,910
of all of the input values that it's getting.
392
392
00:14:48,910 --> 00:14:49,780
Very simple, right?
393
393
00:14:49,780 --> 00:14:51,130
It's very very straight forward.
394
394
00:14:51,130 --> 00:14:53,514
Just add up, multiply by the weight, add them up.
395
395
00:14:53,514 --> 00:14:57,090
And then, it applies an activation function.
396
396
00:14:57,090 --> 00:14:59,340
Now we're gonna talk more about activation function
397
397
00:14:59,340 --> 00:15:00,770
further down but it's basically a function
398
398
00:15:00,770 --> 00:15:03,303
that is assigned to this neuron or to this olier,
399
399
00:15:05,253 --> 00:15:09,460
and it is applied to this weighted sum,
400
400
00:15:09,460 --> 00:15:12,200
and then from that the neuron understands
401
401
00:15:13,640 --> 00:15:16,130
if it needs to pass on a signal.
402
402
00:15:16,130 --> 00:15:18,400
That's the signal it passes on,
403
403
00:15:18,400 --> 00:15:22,220
the function applied to, the weighted sum.
404
404
00:15:22,220 --> 00:15:23,870
But basically depending on the function,
405
405
00:15:23,870 --> 00:15:25,400
the neuron will either pass on the signal or
406
406
00:15:25,400 --> 00:15:27,770
it won't pass the signal on.
407
407
00:15:27,770 --> 00:15:30,343
And that's exactly what happen here in step three.
408
408
00:15:31,320 --> 00:15:33,040
The neuron passes on that signal
409
409
00:15:33,040 --> 00:15:35,720
to the next neuron down the line.
410
410
00:15:35,720 --> 00:15:37,150
And that's what we're going to talk about
411
411
00:15:37,150 --> 00:15:38,680
in the next tutorial because it is
412
412
00:15:38,680 --> 00:15:39,860
quite an important topic.
413
413
00:15:39,860 --> 00:15:41,970
We want to delve deeper into
414
414
00:15:42,810 --> 00:15:43,900
the activation function.
415
415
00:15:43,900 --> 00:15:45,160
But hopefully for now,
416
416
00:15:45,160 --> 00:15:46,940
everything is, should be pretty clear,
417
417
00:15:46,940 --> 00:15:48,740
how you've got input values,
418
418
00:15:48,740 --> 00:15:49,573
you've got weights,
419
419
00:15:49,573 --> 00:15:50,560
you've got these synapses,
420
420
00:15:50,560 --> 00:15:52,740
you've got something that happens in the neuron,
421
421
00:15:52,740 --> 00:15:54,030
you've got weighted sum
422
422
00:15:54,030 --> 00:15:56,026
and then the activation function applied to them
423
423
00:15:56,026 --> 00:15:58,130
that is passed on then that is repeated
424
424
00:15:58,130 --> 00:15:59,850
throughout the whole neural network,
425
425
00:15:59,850 --> 00:16:01,533
on and on and on and on.
426
426
00:16:02,730 --> 00:16:04,480
Thousands hundreds of thousands of times
427
427
00:16:04,480 --> 00:16:06,760
depending on how big, how many neurons you have,
428
428
00:16:06,760 --> 00:16:09,450
how many synapses you have in your neural network.
429
429
00:16:09,450 --> 00:16:10,283
So there we go!
430
430
00:16:10,283 --> 00:16:12,030
Hope you enjoyed today's tutorial,
431
431
00:16:12,030 --> 00:16:13,320
can't wait to see you next time.
432
432
00:16:13,320 --> 00:16:15,120
And until then, enjoy Deep Learning!
36636
Can't find what you're looking for?
Get subtitles in any language from opensubtitles.com, and translate them here.