Original subtitles

1

Hello and welcome back to the course and deep learning now that we've seen your own networks in action

2

it's time for us to find out how they learn.

3

So let's get right into it.

4

They are two fundamentally different approaches to getting a program to do what you want it to do.

5

One is hard coded coding where you actually tell the program's specific rules and what outcomes you

6

want.

7

And you just guide it throughout the whole way and you account for all the possible options that the

8

program has to deal with.

9

On the other hand you have neural networks where you create a facility for the program to be able to

10

understand what it needs to do on its own.

11

So you basically create this neural network where you provided inputs you tell it what you want as outputs

12

and then you let it figure everything out on its own.

13

Two fundamentally different approaches and that is something to keep in mind as we go through these

14

tutorials.

15

Our goal is to create this network which then learns on its own.

16

We going to avoid trying to put in the rules and a good example that I can give you right now is this

17

will come further in the course but it's just a very visual example for instance.

18

How do you distinguish between a dog and cat fur on the left side on the process depicted on the left

19

you would program things like the cat's ears have to be like this look out for whiskers look out for

20

this type of nose look out for this type of shape of face look out for these colors you kind of you'd

21

describe all these things and you'd have conditions like if if the ears are pointy than cat if the ears

22

are sloping down and possibly dog and so on.

23

On the other hand for a neural network you just code the neural networks you code the architecture and

24

then you point the neural network at a folder with all these cats and dogs with images of cats and dogs

25

which are already categorized and you tell it OK I've got you I've got some images of cats and dogs

26

go and learn what a cat is.

27

Go and learn what a dog is.

28

And the neural network will on its own understand everything it needs to understand and then further

29

down once its trained up when you give it a new image of a cat or dog it will be able to understand

30

what it was.

31

So there they are those are the two fundamentally different approaches.

32

And today we're going to slowly start getting into how that second approach works.

33

All right.

34

So let's get straight to it.

35

Here we have a very basic neural network with a one layer is called a single layer feedforward neural

36

network and it is also called a perception.

37

Now before we proceed one thing that we do need to adjust is that output value.

38

Right now you can see that it's just a Y.

39

We need to put a y hat in there.

40

And the reason for that is usually y stands for the actual value and that's what we're going to be using.

41

So why is it going to be the actual value which we see inreality output value is the predicted value

42

by the algorithm by the neural network.

43

Why what is the output value.

44

Basically that's the denomination for the output value.

45

And the perception that was first invented in 1957 by Frank Rosenblat and his whole idea was to create

46

something that can actually learn and adjust itself.

47

And this is what we're going to be looking at now.

48

So we've got our precept drawn.

49

Let's see how our perception learns.

50

So let's say we have some input values that have been supplied to the perception and or basically to

51

our own network.

52

Then the activation function is applied.

53

We have an output and now we're going to plot the output on a chart.

54

So there it is our output y hat.

55

Now what we need to do is in order to be able to learn we need to compare the output value to the actual

56

value that we want the neural network to get right.

57

And that is the value y.

58

And so if we put it here you'll see that there's a bit of a difference.

59

Now we're going to calculate a function called the cost function is calculated as one half of the difference

60

of the square difference between the actual value and output value.

61

Now there there are many ways you can come up for class function.

62

There are many different cost functions that you can use.

63

This is probably the most commonly used call function and why it is specifically this function that

64

we use will find out further down when we're talking about a gradient decent but for now we're just

65

going to agree that this is the cost function and basically what the cost function is telling us is

66

what is the error that you have in your prediction.

67

And our goal is to minimize the cost function because the lower the cost function the closer the y hat

68

is to y.

69

OK so as only we agree on that let's proceed.

70

So basically from here what happens is there is a cost function and from here what happens is now we're

71

going to once we've compared now we're going to feed this information back into the neural network.

72

So there we go there's the information going back into the neural network and it goes to the weights

73

and the weights get updated.

74

Basically the only thing that we have control of in this very simple neural network are the weights

75

w 1 W2 all the way to W..

76

And our goal is to minimize the cost function so all we can do is update the weights.

77

So we update the weights and tweak them a little bit.

78

And how exactly we'll find out for the down but for now we agree that we have the the weights and then

79

we continue so.

80

But here I've put up this screenshot of the data just to make some one point very clear that right now

81

throughout this whole experiment everything we're doing right now we're dealing with just the one role.

82

So we're dealing with we have a dataset of one row where we have for instance we're dealing with how

83

long you study it like the variable that we're predicting is what.

84

What's the result you're going to get on an exam.

85

And the dependent independent variables that we have is how many hours did you study for how many hours

86

did you sleep and what did you get on the quiz.

87

In the mid-semester So in the middle of the semester is a quiz what percentage did you get there.

88

So based on those variables we're trying to predict what score you'll get for the exam and exam the

89

93 percent that's the actual value.

90

So that's why.

91

So.

92

So we feed these three values into a neural network again for the second time now and then we're going

93

to be comparing the result to white.

94

So let's see how this works.

95

We feed these values into the neural network.

96

Everything gets adjusted and weights get it just so as you can see this is again we feed the values

97

again the point here is that we're feeding in the same ball so we only have one roll we're trying to

98

do we're training on one row.

99

This is because this is just a very simple basic example.

100

Then we'll see what happens when there's morals.

101

So again we feed these rows in our cross-functional get adjusted.

102

As you can see everything happens along those lines again.

103

So as you say every time our white hat is changing because we've tweaked the weights.

104

All I had is changing our clothes function changing this whole look again so we feed those in.

105

Why had is changing clothes function is changing.

106

We get information back feedback to the weights so that the weights get adjusted again.

107

We feed in the same values every time everything gets adjusted goes back to the weights.

108

And one more time feed in.

109

OK.

110

And another time so we've adjust the way that just the way we feel in the information.

111

And there we go.

112

So now this time the white hat is equal to y cross-functional 0.

113

Usually you won't get cost function equal to zero.

114

But this is a very simple example.

115

So hopefully all that made sense every time we feed in exactly that same row because just in this case

116

we're just dealing with that one row into our neural network.

117

Well then the weights get the values get valid supply supply the ways activation function is applied

118

we get y hat y had as compared to Y then we see how the cost function is changed.

119

Feedback and the feed that information Bakker's on your own network and then just adjust the weights

120

again.

121

And then we repeat the same process again with the same exact row.

122

We're trying to minimize that cost.

123

So up until now we've been dealing with just that one row.

124

Let's see what happens when you have multiple roles.

125

So here's the full data set.

126

We have eight rows of how many hours you slept or maybe these are different students in day taking the

127

same exam how many other hours they studied how many hours they slept before the exam would get on the

128

quiz and their final result on the test.

129

And as you can see here on the left I've got eight of these perceptions actually.

130

They are all the same perception so this is also important.

131

I just multiplied it or like duplicated eight times just so that we can.

132

Conception is that.

133

But the important thing here is the same neural network we're going to be feeding these into one Samual

134

network.

135

So let's go let's get started.

136

So one airport as you'll hear had lain mentioning one airpark is when we go through a whole dataset

137

and we train our neural network on on all of these roles so those lists are.

138

So there's our first row and there's Why had for the first row there's a second role there's why I had

139

for the second round.

140

So again it's being fed into the same neural network every time.

141

I just copied them several times so we can visually see how this is happening.

142

Then again as it's happening again that's third row fourth row there is our white head for the fourth

143

row and so on.

144

Basically then we get the same values for the remaining four rows as well.

145

So every time we just feed in a row into our neural network we get about it.

146

Then we compare to the actual value.

147

So they are the actual values.

148

So for every single roll we have an actual value.

149

And now based on all of these differences between y hat and why we can calculate the cost function which

150

is the sum of all of those squared differences between why and why and how all of that is halved.

151

And there's our cost function.

152

And basically now what we do after we have the full cost function we go back and we update the weights

153

we update a W 1 WTW.

154

And the important thing to remember here is that all of these perception's all of these neural networks

155

is actually one neural network.

156

So there's not eight of them there's just one.

157

And when we update the weights we're going to update the weights in that one neural network so basically

158

the weights are going to be the same for all of the rows.

159

So it's not the case that every role has its own weight.

160

Now all the rows share the weights and so that's why we looked at the cost function which is the sum

161

of the square differences and then we updated the weights and now from here there was just one iteration.

162

Next we're going to run this whole thing again.

163

We're going to feed every single row into the neural network find out our cost function and do this

164

whole process again.

165

So just as we saw previously where we had just one row and we were doing everything again and again

166

and again same thing here.

167

But now we're going to be doing and Pedros or 800 rows or eight thousand rows however many rows you

168

have in your data set.

169

You do this process and then you calculate the cost function.

170

And the goal here is to minimize the cost function and to get as soon as you found a minute of the cost

171

function that is your final neural network that means your weights have been adjusted and you have found

172

the optimal weights for this dataset that you began your training on and you're ready to proceed to

173

the testing phase or to the application phase.

174

And this whole process is called back propagation.

175

So some additional reading that you might want to do for the cost function and I know we just talked

176

about one and there are many different ones.

177

A good article is located on cross validated.

178

It's called a list of course functions used in neural networks alongside applications.

179

So the euro is there but you can just google for that exact search term or search phrase and you will

180

that this one will be the first one that pops up.

181

It's actually got some good examples and application or use cases for different cost functions so if

182

you're interested to learn more about cost functions Check out this article.

183

And on that note I hope you enjoy this tutorial.

184

I look forward to see you next time.

185

Until then enjoy deep learning.