Would you like to inspect the original subtitles? These are the user uploaded subtitles that are being translated: 1 1 00:00:00,190 --> 00:00:01,400 Hello and Welcome back to the course 2 2 00:00:01,400 --> 00:00:02,320 on Deep Learning. 3 3 00:00:02,320 --> 00:00:03,850 Today we're talking about the neuron, 4 4 00:00:03,850 --> 00:00:06,180 which is the basic building block 5 5 00:00:06,180 --> 00:00:07,880 of artificial neural networks. 6 6 00:00:07,880 --> 00:00:09,280 So let's get started. 7 7 00:00:09,280 --> 00:00:11,220 Previously we saw an image which looked like this. 8 8 00:00:11,220 --> 00:00:13,353 And these are actual, real life neurons 9 9 00:00:13,353 --> 00:00:17,820 which have smeared on to glass, colored a little bit, 10 10 00:00:17,820 --> 00:00:19,880 and they are observed through a microscope. 11 11 00:00:19,880 --> 00:00:21,734 So this is what they look like as you can see, 12 12 00:00:21,734 --> 00:00:23,570 quite an interesting structure. 13 13 00:00:23,570 --> 00:00:27,690 A body, and a lot of different tails, 14 14 00:00:27,690 --> 00:00:30,200 kind of branches coming out of them. 15 15 00:00:30,200 --> 00:00:33,420 And this is very interesting but the question is 16 16 00:00:33,420 --> 00:00:36,040 how can we recreate that in a machine? 17 17 00:00:36,040 --> 00:00:38,900 Because we really need to recreate that in a machine 18 18 00:00:38,900 --> 00:00:41,960 since the whole purpose of Deep Learning is to 19 19 00:00:41,960 --> 00:00:44,523 mimic how the human brain works. 20 20 00:00:46,030 --> 00:00:48,560 In the hopes that by doing so, 21 21 00:00:48,560 --> 00:00:50,890 we are going to create something amazing. 22 22 00:00:50,890 --> 00:00:53,110 We are going to create an amazing infrastructure 23 23 00:00:53,110 --> 00:00:55,110 for machines to be able to learn. 24 24 00:00:55,110 --> 00:00:56,740 And why do we hope for that? 25 25 00:00:56,740 --> 00:00:59,200 Well because the human brain is, 26 26 00:00:59,200 --> 00:01:01,660 well just happens to be one of the most powerful 27 27 00:01:01,660 --> 00:01:04,490 learning tools on the planet, 28 28 00:01:04,490 --> 00:01:07,220 or like learning mechanisms on the planet. 29 29 00:01:07,220 --> 00:01:09,710 And we just hope that if we recreate that 30 30 00:01:09,710 --> 00:01:11,250 we'll have something as awesome as that. 31 31 00:01:11,250 --> 00:01:12,870 So our challenge right now, 32 32 00:01:12,870 --> 00:01:14,410 our very first step to creating 33 33 00:01:14,410 --> 00:01:16,100 artificial neural networks, 34 34 00:01:16,100 --> 00:01:18,270 is to recreate a neuron. 35 35 00:01:18,270 --> 00:01:19,103 So how do we do that? 36 36 00:01:19,103 --> 00:01:21,300 Well, first let's take a closer look 37 37 00:01:21,300 --> 00:01:23,187 at what it actually is. 38 38 00:01:23,187 --> 00:01:26,070 This image was first created by 39 39 00:01:26,070 --> 00:01:27,100 a Spanish neural scientist, 40 40 00:01:27,100 --> 00:01:31,450 Santiago Ramón y Cajal, in 1899. 41 41 00:01:33,080 --> 00:01:36,900 And what he did was he dyed neurons in 42 42 00:01:36,900 --> 00:01:37,930 actual brain tissue and looked 43 43 00:01:37,930 --> 00:01:39,770 at them under a microscope. 44 44 00:01:39,770 --> 00:01:41,020 And while he was looking at them 45 45 00:01:41,020 --> 00:01:42,430 he actually drew what he saw. 46 46 00:01:42,430 --> 00:01:43,440 And this is what he saw. 47 47 00:01:43,440 --> 00:01:45,570 He saw two neurons or two large neurons 48 48 00:01:45,570 --> 00:01:46,830 over there at the top, 49 49 00:01:46,830 --> 00:01:49,950 which had all these branches coming out of them 50 50 00:01:49,950 --> 00:01:53,960 towards their top parts and then each had a 51 51 00:01:53,960 --> 00:01:58,960 rod or thread coming out towards the bottom, very long one. 52 52 00:01:59,400 --> 00:02:01,550 And that's what he saw. 53 53 00:02:01,550 --> 00:02:03,560 And now, you know, technology has advanced 54 54 00:02:03,560 --> 00:02:07,020 quite a lot and we have seen neurons much closer 55 55 00:02:07,020 --> 00:02:08,969 and more detailed and now we can actually draw 56 56 00:02:08,969 --> 00:02:11,940 what it looks like diagrammatically. 57 57 00:02:11,940 --> 00:02:13,290 So let's have a look at that. 58 58 00:02:13,290 --> 00:02:15,300 Here's a neuron, this is what it looks like. 59 59 00:02:15,300 --> 00:02:20,300 Very similar to what Santiago Ramón drew over here. 60 60 00:02:21,000 --> 00:02:23,010 Here in this neuron what we can see is that 61 61 00:02:23,010 --> 00:02:26,420 line:15% its got a body, that's the main part of the neuron. 62 62 00:02:26,420 --> 00:02:28,050 And then its got some branches at the top, 63 63 00:02:28,050 --> 00:02:29,050 which are called dendrites. 64 64 00:02:29,050 --> 00:02:30,540 And its also got an axon, 65 65 00:02:30,540 --> 00:02:33,380 which is that long tail of the neuron. 66 66 00:02:33,380 --> 00:02:36,620 So what are these dendrites for and what's the axon for. 67 67 00:02:36,620 --> 00:02:39,860 Well, the key point to understand here is that 68 68 00:02:39,860 --> 00:02:44,250 neurons by themselves are pretty much useless. 69 69 00:02:44,250 --> 00:02:45,900 It's like an ant. 70 70 00:02:45,900 --> 00:02:47,970 An ant on its own can't do much, 71 71 00:02:47,970 --> 00:02:51,110 like 5 ants together maybe they can pick something up. 72 72 00:02:51,110 --> 00:02:54,040 But again, they can't build an ant hill, 73 73 00:02:54,040 --> 00:02:55,360 they can't establish a colony, 74 74 00:02:55,360 --> 00:02:59,290 they can't work together as a huge organism. 75 75 00:02:59,290 --> 00:03:01,460 But at the same time, when you have lots and lots of ants, 76 76 00:03:01,460 --> 00:03:04,220 like you have a million ants, they can build a whole colony, 77 77 00:03:04,220 --> 00:03:05,680 they can build an ant hill. 78 78 00:03:05,680 --> 00:03:06,520 Same thing with neurons. 79 79 00:03:06,520 --> 00:03:07,740 By itself it's not that strong, 80 80 00:03:07,740 --> 00:03:09,750 but when you have lots of neurons together, 81 81 00:03:09,750 --> 00:03:12,018 they work together to do magic. 82 82 00:03:12,018 --> 00:03:14,330 And how do they work together? That's a question. 83 83 00:03:14,330 --> 00:03:16,610 Well, that's what the dendrites and axon are for. 84 84 00:03:16,610 --> 00:03:18,370 So the dendrites are kind of like the 85 85 00:03:18,370 --> 00:03:19,960 receivers of the signal for the neuron, 86 86 00:03:19,960 --> 00:03:23,100 and axon is the transmitter of the signal for the neuron. 87 87 00:03:23,100 --> 00:03:26,450 And here's an image of how it all works conceptually. 88 88 00:03:26,450 --> 00:03:27,920 So at the top you got a neuron, 89 89 00:03:27,920 --> 00:03:31,140 and you can see that its dendrites are connected 90 90 00:03:31,140 --> 00:03:33,870 to axons of other neurons that are like 91 91 00:03:33,870 --> 00:03:35,790 even further away above it. 92 92 00:03:35,790 --> 00:03:38,880 And then the signal from this neuron travels down 93 93 00:03:38,880 --> 00:03:41,726 its axon and connects or passes onto 94 94 00:03:41,726 --> 00:03:43,530 the dendrites of the other neuron. 95 95 00:03:43,530 --> 00:03:44,930 And that's how they're connected. 96 96 00:03:44,930 --> 00:03:46,670 And in that small image over there, 97 97 00:03:46,670 --> 00:03:47,730 you can see that 98 98 00:03:47,730 --> 00:03:51,893 the axon doesn't actually touch the dendrite. 99 99 00:03:52,764 --> 00:03:54,810 (laughs) A lot of machine learning, 100 100 00:03:54,810 --> 00:03:56,674 or a few machine learning scientists 101 101 00:03:56,674 --> 00:03:59,100 are very adamant about the fact 102 102 00:03:59,100 --> 00:04:00,200 that it doesn't touch. 103 103 00:04:02,690 --> 00:04:04,680 It doesn't touch, it has been proven that 104 104 00:04:04,680 --> 00:04:06,870 there is no physical connection there. 105 105 00:04:06,870 --> 00:04:09,080 But the point that we are interested in 106 106 00:04:09,080 --> 00:04:11,200 is that that connection between them, 107 107 00:04:11,200 --> 00:04:15,160 that the whole concept of the signal being passed, 108 108 00:04:15,160 --> 00:04:16,220 that's called the synapse. 109 109 00:04:16,220 --> 00:04:18,470 line:15% You can see over there, in that little image, 110 110 00:04:20,041 --> 00:04:22,610 line:15% that figure bracket is synapse. 111 111 00:04:22,610 --> 00:04:23,880 That's the term we're going to be using. 112 112 00:04:23,880 --> 00:04:27,830 Instead of calling our artificial neurons, 113 113 00:04:27,830 --> 00:04:28,950 the lines we're gonna have, 114 114 00:04:28,950 --> 00:04:30,580 or the connectors for artificial neurons 115 115 00:04:30,580 --> 00:04:32,517 we're not be calling them axons or dendrites, 116 116 00:04:32,517 --> 00:04:34,110 because then the question is 117 117 00:04:34,110 --> 00:04:35,120 whose connection is this? 118 118 00:04:35,120 --> 00:04:36,070 Is it that neuron's or is it this neuron's? 119 119 00:04:36,070 --> 00:04:39,055 We're just going to call them synapses. 120 120 00:04:39,055 --> 00:04:42,721 And that kind of just answers all the questions. 121 121 00:04:42,721 --> 00:04:45,050 I mean it's basically just where the signal is passed. 122 122 00:04:45,050 --> 00:04:47,097 Doesn't matter who that element belongs to. 123 123 00:04:47,097 --> 00:04:50,005 That's just a representation of the signal 124 124 00:04:50,005 --> 00:04:51,870 being passed and we see that just now. 125 125 00:04:51,870 --> 00:04:54,773 So basically that's how a neuron works. 126 126 00:04:56,340 --> 00:04:59,653 Let's move on to how we're going to represent 127 127 00:04:59,653 --> 00:05:02,503 neurons or how we're going to create neurons in machines. 128 128 00:05:03,665 --> 00:05:06,330 So now we're moving away from neural science 129 129 00:05:06,330 --> 00:05:09,280 and moving into technology. 130 130 00:05:09,280 --> 00:05:10,250 And here we go. 131 131 00:05:10,250 --> 00:05:11,590 So, here's our neuron, 132 132 00:05:11,590 --> 00:05:13,640 also sometimes called the node. 133 133 00:05:13,640 --> 00:05:15,938 The neuron gets some input signals. 134 134 00:05:15,938 --> 00:05:18,290 And it has an output signal. 135 135 00:05:18,290 --> 00:05:20,930 So dendrites and axons, remember? 136 136 00:05:20,930 --> 00:05:23,083 But again, we're gonna call these synopses. 137 137 00:05:25,860 --> 00:05:26,693 These input signals, 138 138 00:05:26,693 --> 00:05:27,900 we're going to represent them with 139 139 00:05:27,900 --> 00:05:28,980 other neurons as well. 140 140 00:05:28,980 --> 00:05:30,690 So, in this specific case, 141 141 00:05:30,690 --> 00:05:31,523 you can see that 142 142 00:05:31,523 --> 00:05:33,760 this neuron, this green neuron, 143 143 00:05:33,760 --> 00:05:35,780 is getting signals from yellow neurons. 144 144 00:05:35,780 --> 00:05:37,410 And in this course, we are going to try 145 145 00:05:37,410 --> 00:05:40,530 to stick to a certain color coding regime, 146 146 00:05:40,530 --> 00:05:42,470 where yellow means an input layer. 147 147 00:05:42,470 --> 00:05:45,540 So basically all the neurons that are 148 148 00:05:45,540 --> 00:05:48,780 on the outer layer, on the first front of 149 149 00:05:49,750 --> 00:05:51,540 where the signals coming in. 150 150 00:05:51,540 --> 00:05:54,990 By signal, it might be a bit of an overkill 151 151 00:05:56,030 --> 00:05:57,040 to call this a signal. 152 152 00:05:57,040 --> 00:05:58,680 It's just basically input value. 153 153 00:05:58,680 --> 00:06:01,180 So you know how even like in a simple 154 154 00:06:01,180 --> 00:06:03,211 linear regression you have input values, 155 155 00:06:03,211 --> 00:06:04,950 and then you have a predicted value. 156 156 00:06:04,950 --> 00:06:05,783 Same thing here. 157 157 00:06:05,783 --> 00:06:07,120 So you have input values, 158 158 00:06:07,120 --> 00:06:08,940 and there they are, the yellow ones. 159 159 00:06:08,940 --> 00:06:10,630 And on the right to you we see just now 160 160 00:06:10,630 --> 00:06:12,578 it'll be red, it'll be the output value. 161 161 00:06:12,578 --> 00:06:15,620 The thing that I wanted to point out here is that 162 162 00:06:15,620 --> 00:06:17,010 in this specific example we are looking at 163 163 00:06:17,010 --> 00:06:19,660 a neuron which is getting its signals from 164 164 00:06:19,660 --> 00:06:21,220 the input layer neurons. 165 165 00:06:21,220 --> 00:06:22,460 So they are also neurons but 166 166 00:06:22,460 --> 00:06:23,920 they are input layer neurons. 167 167 00:06:23,920 --> 00:06:26,590 Sometimes you'll have neurons which 168 168 00:06:26,590 --> 00:06:30,420 get their signal from other hidden layer neurons, 169 169 00:06:30,420 --> 00:06:31,670 so from other green neurons. 170 170 00:06:31,670 --> 00:06:32,933 And the concept is gonna be exactly the same. 171 171 00:06:32,933 --> 00:06:35,950 Just in this case, for simplicity's sake, 172 172 00:06:35,950 --> 00:06:37,510 we're portraying this example. 173 173 00:06:37,510 --> 00:06:38,960 And in terms of the input layer, 174 174 00:06:38,960 --> 00:06:40,360 the way to think about it is 175 175 00:06:42,990 --> 00:06:44,857 in the analogy of the human brain, 176 176 00:06:44,857 --> 00:06:48,030 the input layer is your senses, right. 177 177 00:06:48,030 --> 00:06:49,810 So whatever you can see, hear, 178 178 00:06:49,810 --> 00:06:52,380 feel, touch or smell. 179 179 00:06:52,380 --> 00:06:53,340 And of course, 180 180 00:06:53,340 --> 00:06:55,760 there's a lot of things you can see, 181 181 00:06:55,760 --> 00:06:57,670 there's a lot of information coming in. 182 182 00:06:57,670 --> 00:06:58,780 But those are your... 183 183 00:06:58,780 --> 00:07:00,040 that's what your brain is limited to, 184 184 00:07:00,040 --> 00:07:02,880 it's pretty much a (laughs) 185 185 00:07:02,880 --> 00:07:04,770 it's pretty much lives in a box 186 186 00:07:04,770 --> 00:07:07,690 made out of bones and it's only... 187 187 00:07:07,690 --> 00:07:09,370 It's a mind blowing fact to think about. 188 188 00:07:09,370 --> 00:07:13,080 Your brain is just locked in a black box, 189 189 00:07:13,080 --> 00:07:13,913 and the only thing... 190 190 00:07:13,913 --> 00:07:15,350 and it can't see, it can't hear, 191 191 00:07:15,350 --> 00:07:16,210 the only thing it's getting 192 192 00:07:16,210 --> 00:07:18,150 is electrical impulses coming from 193 193 00:07:18,150 --> 00:07:20,410 these organs that you have, 194 194 00:07:20,410 --> 00:07:22,663 which are called your ears, nose, eyes, 195 195 00:07:23,520 --> 00:07:28,520 your sense of touch and whatever... and your taste. 196 196 00:07:28,590 --> 00:07:30,120 It's just getting signals but 197 197 00:07:30,120 --> 00:07:32,090 it basically lives in this dark black box 198 198 00:07:32,090 --> 00:07:35,960 and it's making sense of the world through your senses. 199 199 00:07:35,960 --> 00:07:37,413 It's phenomenal. 200 200 00:07:38,910 --> 00:07:41,537 So you have these inputs that are coming in, 201 201 00:07:41,537 --> 00:07:44,192 and in terms of human brain those are your five senses, 202 202 00:07:44,192 --> 00:07:47,540 in terms of machine learning or deep learning, 203 203 00:07:47,540 --> 00:07:50,530 that is basically your input values, 204 204 00:07:50,530 --> 00:07:52,066 so your independent variables, 205 205 00:07:52,066 --> 00:07:52,899 and we will get to that in a second. 206 206 00:07:52,899 --> 00:07:54,923 So your input values, 207 207 00:07:56,310 --> 00:07:58,780 the signal is passed through synapses to your neuron, 208 208 00:07:58,780 --> 00:08:00,700 and then your neuron has an output value, 209 209 00:08:00,700 --> 00:08:03,440 that it passes further on down the chain. 210 210 00:08:03,440 --> 00:08:05,160 In this specific case, in terms of color coding, 211 211 00:08:05,160 --> 00:08:06,940 again yellow means input layer. 212 212 00:08:06,940 --> 00:08:08,520 So we kind of simplifying everything here. 213 213 00:08:08,520 --> 00:08:11,050 We're saying we're only gonna have like the input layer, 214 214 00:08:11,050 --> 00:08:13,150 then we're gonna have one hidden layer, 215 215 00:08:13,150 --> 00:08:15,010 with the green, which is a hidden layer, 216 216 00:08:15,010 --> 00:08:17,410 and then we're gonna have our output layer right away. 217 217 00:08:17,410 --> 00:08:20,253 So just so that we can get used to those colors for now. 218 218 00:08:21,470 --> 00:08:23,950 So there we go, that's the basic structure. 219 219 00:08:23,950 --> 00:08:26,020 So now let's look at a bit more detail 220 220 00:08:26,020 --> 00:08:28,310 at these different elements that we have. 221 221 00:08:28,310 --> 00:08:29,870 So we got the input layer. 222 222 00:08:29,870 --> 00:08:31,000 And what do we have here? 223 223 00:08:31,000 --> 00:08:33,840 Well, we have these inputs which are 224 224 00:08:33,840 --> 00:08:35,410 in fact independent variables. 225 225 00:08:35,410 --> 00:08:36,360 So independent variable one, 226 226 00:08:36,360 --> 00:08:37,193 independent variable two, 227 227 00:08:37,193 --> 00:08:38,570 and independent variable m. 228 228 00:08:38,570 --> 00:08:39,880 The important thing to remember here, 229 229 00:08:39,880 --> 00:08:42,700 is that these independent variables 230 230 00:08:42,700 --> 00:08:44,650 are all for one single observation. 231 231 00:08:44,650 --> 00:08:47,530 So think of it as one row in your data base. 232 232 00:08:47,530 --> 00:08:48,930 One observation. 233 233 00:08:48,930 --> 00:08:51,303 You just take all of the independent variables, 234 234 00:08:52,158 --> 00:08:54,841 maybe it's the age of the person, 235 235 00:08:54,841 --> 00:08:57,727 the amount of money in their bank account, 236 236 00:08:57,727 --> 00:09:00,690 how do they drive or walk to work, 237 237 00:09:00,690 --> 00:09:02,835 what method of transportation do they use. 238 238 00:09:02,835 --> 00:09:06,030 But that's all descriptions of one specific person, 239 239 00:09:06,030 --> 00:09:08,970 that you are, either you're training your model on, 240 240 00:09:08,970 --> 00:09:11,724 or you're performing some prediction on. 241 241 00:09:11,724 --> 00:09:14,010 And the other thing you need to know 242 242 00:09:14,010 --> 00:09:15,050 about these variables is that 243 243 00:09:15,050 --> 00:09:16,700 you need to standardize them. 244 244 00:09:16,700 --> 00:09:19,140 You need to either standardize them which means 245 245 00:09:19,140 --> 00:09:21,250 make sure they have a mean of zero and variance one, 246 246 00:09:21,250 --> 00:09:23,259 or you can also sometimes and 247 247 00:09:23,259 --> 00:09:25,840 Hadelin will point out these tricks 248 248 00:09:25,840 --> 00:09:27,960 in a bit more detail, 249 249 00:09:27,960 --> 00:09:29,730 perhaps in the practical tutorials 250 250 00:09:29,730 --> 00:09:30,890 you might come across these, 251 251 00:09:30,890 --> 00:09:32,880 sometimes you might want to not standardize 252 252 00:09:32,880 --> 00:09:34,900 you might wanna normalize them. 253 253 00:09:34,900 --> 00:09:37,264 Meaning that instead of making sure that 254 254 00:09:37,264 --> 00:09:38,900 mean is zero and variance is one, 255 255 00:09:38,900 --> 00:09:42,510 you just subtract the minimum value and 256 256 00:09:42,510 --> 00:09:44,450 then you divide it by maximum minus minimum, 257 257 00:09:44,450 --> 00:09:46,187 so by the range of your values and 258 258 00:09:46,187 --> 00:09:49,067 therefore you get values between zero and one. 259 259 00:09:51,190 --> 00:09:53,030 Depend on the scenario you might wanna do one 260 260 00:09:53,030 --> 00:09:54,250 or the other but basically you want 261 261 00:09:54,250 --> 00:09:56,880 all of these variables to be quite similar, 262 262 00:09:56,880 --> 00:10:01,200 in about the same range of values. 263 263 00:10:01,200 --> 00:10:02,233 Why's that? 264 264 00:10:03,483 --> 00:10:04,316 Well all of these values are going to 265 265 00:10:04,316 --> 00:10:05,640 go into a neural network where as 266 266 00:10:05,640 --> 00:10:07,842 we all see just now they will be added up and 267 267 00:10:07,842 --> 00:10:10,280 multiplied by weights added up and so on. 268 268 00:10:10,280 --> 00:10:13,020 It's just going to be easier for 269 269 00:10:13,020 --> 00:10:14,763 the neural network to process them 270 270 00:10:14,763 --> 00:10:16,763 if they are all about the same. 271 271 00:10:19,271 --> 00:10:21,729 And that's just how it is going to 272 272 00:10:21,729 --> 00:10:24,120 be able to work properly. 273 273 00:10:24,120 --> 00:10:26,280 And if you want to read more about 274 274 00:10:26,280 --> 00:10:28,870 standardization, normalization and other things 275 275 00:10:28,870 --> 00:10:30,560 you can do with your input variables, 276 276 00:10:30,560 --> 00:10:34,650 a good additional reading paper is called 277 277 00:10:34,650 --> 00:10:38,450 Efficient BackProp by Yan LeCun 1998, 278 278 00:10:38,450 --> 00:10:40,297 line:15% the link's over there. 279 279 00:10:40,297 --> 00:10:41,430 So Yan LeCun, we're actually going to 280 280 00:10:41,430 --> 00:10:44,720 talk about this phenomenal person 281 281 00:10:44,720 --> 00:10:45,850 in the place of Deep Learning 282 282 00:10:45,850 --> 00:10:48,240 in the part of the course where 283 283 00:10:48,240 --> 00:10:50,616 we're talking about illusional neural networks. 284 284 00:10:50,616 --> 00:10:53,180 You'll see that this is definitely a person 285 285 00:10:53,180 --> 00:10:55,180 who knows what he's talking about. 286 286 00:10:55,180 --> 00:10:57,630 He's a close friend of Geoffrey Hinton, 287 287 00:10:57,630 --> 00:11:00,790 who we already seen, who we've already mentioned. 288 288 00:11:00,790 --> 00:11:04,040 So in this paper you will learn more about 289 289 00:11:04,040 --> 00:11:05,590 standardization and normalization. 290 290 00:11:05,590 --> 00:11:07,070 But you can pick up lots of 291 291 00:11:07,070 --> 00:11:09,000 other different tips and tricks and 292 292 00:11:09,000 --> 00:11:11,180 be a good source of additional reading 293 293 00:11:11,180 --> 00:11:12,490 as you go through this course. 294 294 00:11:12,490 --> 00:11:14,010 So check it out if you're interested 295 295 00:11:14,010 --> 00:11:16,190 in some additional reading. 296 296 00:11:16,190 --> 00:11:18,590 There we go, so that's what we need to do 297 297 00:11:18,590 --> 00:11:20,280 with the variables. 298 298 00:11:20,280 --> 00:11:23,090 And here we've got the output value. 299 299 00:11:23,090 --> 00:11:25,010 So what can our output value be? 300 300 00:11:25,010 --> 00:11:26,220 Well we've got a couple of options. 301 301 00:11:26,220 --> 00:11:27,950 Output value can be, 302 302 00:11:27,950 --> 00:11:30,040 it can be continuous, for instance, price; 303 303 00:11:30,040 --> 00:11:31,550 it can be binary, for instance, 304 304 00:11:31,550 --> 00:11:33,610 a person will exit or stay; 305 305 00:11:33,610 --> 00:11:35,913 or it can be a categorical variable. 306 306 00:11:37,160 --> 00:11:39,180 If it's a categorical variable, 307 307 00:11:39,180 --> 00:11:41,010 the important thing to remember here is that 308 308 00:11:41,010 --> 00:11:43,790 in that case, your output value won't be just one, 309 309 00:11:43,790 --> 00:11:45,920 it'll be several output values, 310 310 00:11:45,920 --> 00:11:47,920 because these will be your dummy variables, 311 311 00:11:47,920 --> 00:11:51,260 which will be representing your categories. 312 312 00:11:51,260 --> 00:11:53,193 And that's just how it works. 313 313 00:11:54,110 --> 00:11:55,400 Just important to remember that, 314 314 00:11:55,400 --> 00:11:58,320 in that case that's how you're going to be getting 315 315 00:11:58,320 --> 00:12:02,243 your categories out of the artificial neural network. 316 316 00:12:02,243 --> 00:12:04,170 But let's go back to our simple case 317 317 00:12:04,170 --> 00:12:05,640 of one output value. 318 318 00:12:05,640 --> 00:12:10,080 And now one more point, a point I've already made, 319 319 00:12:10,080 --> 00:12:12,533 I just want to reiterate this point. 320 320 00:12:12,533 --> 00:12:15,360 On the left you've got a single observation, 321 321 00:12:15,360 --> 00:12:17,630 so one row from your data set, 322 322 00:12:17,630 --> 00:12:19,850 and on the right you have a single observation as well. 323 323 00:12:19,850 --> 00:12:22,080 That is the same observation. 324 324 00:12:22,080 --> 00:12:25,420 So important to remember that whatever inputs 325 325 00:12:25,420 --> 00:12:27,370 you're putting in, that's for one row, 326 326 00:12:27,370 --> 00:12:28,520 and then the output you get back is 327 327 00:12:28,520 --> 00:12:30,000 for that exact same row. 328 328 00:12:30,000 --> 00:12:32,490 Or if you're training your neural network then 329 329 00:12:32,490 --> 00:12:34,380 you're putting the inputs in for that one row, 330 330 00:12:34,380 --> 00:12:36,680 you're putting the output in for that one row. 331 331 00:12:37,967 --> 00:12:39,250 So if you wanna simplify the complexity, 332 332 00:12:39,250 --> 00:12:42,170 think of it as like a simple linear regression, 333 333 00:12:42,170 --> 00:12:43,900 or a multi-variant linear regression. 334 334 00:12:43,900 --> 00:12:46,270 So you're putting in your values, 335 335 00:12:46,270 --> 00:12:47,583 you have your output. 336 336 00:12:47,583 --> 00:12:49,610 There's no question about it 337 337 00:12:49,610 --> 00:12:51,660 when we are talking about things like regression, 338 338 00:12:51,660 --> 00:12:52,700 because we're so used to it. 339 339 00:12:52,700 --> 00:12:54,890 Same thing here. It's nothing too complex. 340 340 00:12:54,890 --> 00:12:56,030 We're just putting in values, 341 341 00:12:56,030 --> 00:12:56,863 we're getting an output. 342 342 00:12:56,863 --> 00:12:58,270 But just remember that every time 343 343 00:12:58,270 --> 00:12:59,500 it's one row that you're dealing with. 344 344 00:12:59,500 --> 00:13:01,730 So you don't get confused and start putting in 345 345 00:13:01,730 --> 00:13:05,400 like thinking these are different rows that 346 346 00:13:05,400 --> 00:13:07,860 you're putting into your artificial 347 347 00:13:07,860 --> 00:13:09,060 neural network or something. 348 348 00:13:09,060 --> 00:13:11,170 This is all just values in that one row. 349 349 00:13:11,170 --> 00:13:12,240 So different observation, 350 350 00:13:12,240 --> 00:13:13,900 different characteristics of, 351 351 00:13:13,900 --> 00:13:16,730 or attributes relating to that one observation. 352 352 00:13:16,730 --> 00:13:17,630 Every single time. 353 353 00:13:19,070 --> 00:13:20,280 Okay so next thing that we wanna 354 354 00:13:20,280 --> 00:13:24,740 talk about here is the synapses. 355 355 00:13:24,740 --> 00:13:25,890 Here we've got synapses and 356 356 00:13:25,890 --> 00:13:28,840 they all actually get assigned weights. 357 357 00:13:28,840 --> 00:13:31,740 We're gonna talk more about weights further down, 358 358 00:13:31,740 --> 00:13:36,500 but in short, weights are crucial to 359 359 00:13:36,500 --> 00:13:38,560 artificial neural networks functioning. 360 360 00:13:38,560 --> 00:13:41,710 Because weights are how neural networks learn. 361 361 00:13:41,710 --> 00:13:44,320 By adjusting the weights, 362 362 00:13:44,320 --> 00:13:47,340 the neural network decides in every single case, 363 363 00:13:47,340 --> 00:13:49,490 what signal is important and what signal 364 364 00:13:49,490 --> 00:13:51,050 is not important to a certain neuron, 365 365 00:13:51,050 --> 00:13:52,420 what signal gets passed along and 366 366 00:13:52,420 --> 00:13:54,250 what signal doesn't get passed along, 367 367 00:13:54,250 --> 00:13:56,240 or to what strength, to what extent 368 368 00:13:56,240 --> 00:13:57,670 signals get passed along. 369 369 00:13:57,670 --> 00:13:59,240 So weights are crucial, 370 370 00:13:59,240 --> 00:14:02,390 they are the things that get adjusted 371 371 00:14:02,390 --> 00:14:04,100 through the process of learning. 372 372 00:14:04,100 --> 00:14:06,100 When you're training your artificial neural network, 373 373 00:14:06,100 --> 00:14:08,450 you're basically adjusting all of the weights 374 374 00:14:08,450 --> 00:14:09,940 in all of the synapses across this 375 375 00:14:09,940 --> 00:14:11,310 whole neural network and 376 376 00:14:11,310 --> 00:14:13,660 that's where gradient descent and 377 377 00:14:15,010 --> 00:14:16,770 back propagation come into play and 378 378 00:14:16,770 --> 00:14:19,550 those are concepts that we'll also discuss. 379 379 00:14:19,550 --> 00:14:21,310 So basically those are the weights. 380 380 00:14:21,310 --> 00:14:23,710 That's all you need to know for now. 381 381 00:14:23,710 --> 00:14:24,543 Here we've got the neuron. 382 382 00:14:24,543 --> 00:14:26,960 So signals go into the neuron and 383 383 00:14:26,960 --> 00:14:28,340 what happens in the neuron? 384 384 00:14:28,340 --> 00:14:30,690 So this is the interesting part. 385 385 00:14:30,690 --> 00:14:32,150 We're talking about the neuron today, 386 386 00:14:32,150 --> 00:14:33,800 what happens inside the neuron? 387 387 00:14:33,800 --> 00:14:35,450 So, a few things happen. 388 388 00:14:35,450 --> 00:14:37,980 First thing, and the first step is that 389 389 00:14:37,980 --> 00:14:41,260 all of these values that it's getting, get added up. 390 390 00:14:41,260 --> 00:14:45,430 So it takes the added, so the weighted sum 391 391 00:14:45,430 --> 00:14:48,910 of all of the input values that it's getting. 392 392 00:14:48,910 --> 00:14:49,780 Very simple, right? 393 393 00:14:49,780 --> 00:14:51,130 It's very very straight forward. 394 394 00:14:51,130 --> 00:14:53,514 Just add up, multiply by the weight, add them up. 395 395 00:14:53,514 --> 00:14:57,090 And then, it applies an activation function. 396 396 00:14:57,090 --> 00:14:59,340 Now we're gonna talk more about activation function 397 397 00:14:59,340 --> 00:15:00,770 further down but it's basically a function 398 398 00:15:00,770 --> 00:15:03,303 that is assigned to this neuron or to this olier, 399 399 00:15:05,253 --> 00:15:09,460 and it is applied to this weighted sum, 400 400 00:15:09,460 --> 00:15:12,200 and then from that the neuron understands 401 401 00:15:13,640 --> 00:15:16,130 if it needs to pass on a signal. 402 402 00:15:16,130 --> 00:15:18,400 That's the signal it passes on, 403 403 00:15:18,400 --> 00:15:22,220 the function applied to, the weighted sum. 404 404 00:15:22,220 --> 00:15:23,870 But basically depending on the function, 405 405 00:15:23,870 --> 00:15:25,400 the neuron will either pass on the signal or 406 406 00:15:25,400 --> 00:15:27,770 it won't pass the signal on. 407 407 00:15:27,770 --> 00:15:30,343 And that's exactly what happen here in step three. 408 408 00:15:31,320 --> 00:15:33,040 The neuron passes on that signal 409 409 00:15:33,040 --> 00:15:35,720 to the next neuron down the line. 410 410 00:15:35,720 --> 00:15:37,150 And that's what we're going to talk about 411 411 00:15:37,150 --> 00:15:38,680 in the next tutorial because it is 412 412 00:15:38,680 --> 00:15:39,860 quite an important topic. 413 413 00:15:39,860 --> 00:15:41,970 We want to delve deeper into 414 414 00:15:42,810 --> 00:15:43,900 the activation function. 415 415 00:15:43,900 --> 00:15:45,160 But hopefully for now, 416 416 00:15:45,160 --> 00:15:46,940 everything is, should be pretty clear, 417 417 00:15:46,940 --> 00:15:48,740 how you've got input values, 418 418 00:15:48,740 --> 00:15:49,573 you've got weights, 419 419 00:15:49,573 --> 00:15:50,560 you've got these synapses, 420 420 00:15:50,560 --> 00:15:52,740 you've got something that happens in the neuron, 421 421 00:15:52,740 --> 00:15:54,030 you've got weighted sum 422 422 00:15:54,030 --> 00:15:56,026 and then the activation function applied to them 423 423 00:15:56,026 --> 00:15:58,130 that is passed on then that is repeated 424 424 00:15:58,130 --> 00:15:59,850 throughout the whole neural network, 425 425 00:15:59,850 --> 00:16:01,533 on and on and on and on. 426 426 00:16:02,730 --> 00:16:04,480 Thousands hundreds of thousands of times 427 427 00:16:04,480 --> 00:16:06,760 depending on how big, how many neurons you have, 428 428 00:16:06,760 --> 00:16:09,450 how many synapses you have in your neural network. 429 429 00:16:09,450 --> 00:16:10,283 So there we go! 430 430 00:16:10,283 --> 00:16:12,030 Hope you enjoyed today's tutorial, 431 431 00:16:12,030 --> 00:16:13,320 can't wait to see you next time. 432 432 00:16:13,320 --> 00:16:15,120 And until then, enjoy Deep Learning! 36636