Original subtitles

1

Hello and welcome to this new tutorial.

2

The second big step of this implementation we have already created one brain.

3

The brain of the generator.

4

And now we're about to create the second brain of our deep convolutional Ganns.

5

I'm talking of course about the brain of the discriminator and we're going to proceed exactly the same

6

way as we proceeded for the generator.

7

We're first going to define a class that will define the architecture of the neural network for the

8

discriminator.

9

Then this class will also include a forward function to forward propagate the signal inside this new

10

network.

11

And then once the class is made we'll create a discriminator object that is will create the brain of

12

the discriminator.

13

So let's do this let's tackle the second big step of our deep convolutional gets.

14

So we are starting with Class Of course then we need to give a name to this new class that will define

15

the discriminator.

16

And since we called the class of our generated G We're going to call this one the thing that makes sense.

17

And so d again is going to inherit from the end in module and so inside we use again and then that module.

18

All right then Kallen and then we go inside the class to start with the init function that will define

19

the architecture of the neural network.

20

So that's the same we start with Def and then double underscore in it that will underscore again and

21

some parenthesis.

22

And inside we only put self which will refer to the future instances of our class that will be created.

23

That is the object and that will allow us to associate the specific variables that are attached to the

24

object that is I'm talking about the properties of the object.

25

The object will be the neural network of discriminator.

26

So these properties and variables will be the different modules composing the architecture of the discriminator.

27

All right so now Collen we go inside the function and same we start by activating the inheritance using

28

the super function which takes as arguments are discriminator class and our object self.

29

Then don't forget we had a done here double underscore in it double underscore again.

30

And some parenthesis.

31

Perfect inheritance activated all right and now things are going to get interesting.

32

I'm going to try to make you guess what we're going to start with.

33

We have to start with the first module of this architecture of the neural network of the discriminator.

34

And so according to you what is going to be this first module is it going to be an inverse convolution

35

like before or is it going to be a convolution or is it going to be a fool connection.

36

Well think about this for a second.

37

And until then I will get my main again which represents the metal module that will contain the different

38

modules composing the architecture of the neural network.

39

So exactly like before.

40

So cells that main and this module will be a big sequence of layers.

41

That is a big sequence of modules.

42

And therefore I'm setting this main metal module to be an object of the.

43

And in that sequential class parenthesis.

44

And here we go.

45

Let's see if you got this right.

46

For this first module of this architecture.

47

Let's see.

48

All right so to answer this question right.

49

Well we need to understand what the discriminator is going to do.

50

It's going to take as inputs a generated image coming from the generator and return as output a discriminating

51

number which will be of value between 0 and 1.

52

Therefore since it takes as inputs a generated image the generated image created by the generator.

53

Well that means it takes as input an image and therefore the first module of our architecture is simply

54

going to be a convolution not an inverse convolution.

55

That's only for the generator.

56

But this time a convolution it's going to take as input an image an image created by the generator and

57

through the convolution.

58

And then of course some more convolutions we will get in the end a simple vector of one element that

59

will contain the discriminating number a value between 0 and 1.

60

So there we go.

61

You have the idea and now it's going to be the same as before.

62

We're going to add a series of different convolutions with different parameters and will best the feature

63

maps.

64

And of course we'll apply some rectification.

65

And we'll let you find out about some surprise regarding the rectification.

66

It's not going to be a real it's going to be something a little more sophisticated.

67

We will see that in one or two minutes.

68

But let's start with our first module that is our convolution.

69

So to get this first module we're going to create a new object of the comp to the class and therefore

70

I'm taking my and then module that and then I'm getting the code to the class.

71

Here it is perfect.

72

There is now something less easy.

73

The parameters of this convert to the class.

74

So it's actually not that hard and I'm going to give you something else to get what do you think is

75

going to be the parameter we have to improve here.

76

For the first argument of this comes to the class.

77

Remember that corresponds to in channels that is the dimensions of the inputs.

78

So as we just said the input of our discriminator and therefore of this first module that is the conclusion

79

is an image an image created by the generator.

80

So to answer the question what is going to be the inputs of this first convolution.

81

We simply need to look at the output of the generator and the output of the generator.

82

Remember is these three channels here of the image.

83

So the input of our discriminator is nothing else than three to three channels of the generated image

84

created by the generator.

85

Perfect.

86

And then it's going to be the same.

87

We're going to specify a number of feature maps kernel size astride a padding and choosing true or false

88

for the bias.

89

So here we go let's do it.

90

We're going to choose 64 feature maps then a size of 4 for the Colonel's so.

91

So these are going to be squares of size four by four then astride of 2 and abetting of one.

92

And eventually Last but not least we choose to have no bias and therefore like before I'm choosing by

93

as equals force.

94

All right.

95

First Mudgal done congratulations.

96

Now second module What do you think the second.

97

Well operation is going to be we're not going to apply a budget yet.

98

We will leave that for the next convolution.

99

However we always need to apply rectification and that's the surprise I wanted to keep from you but

100

I'll tell you it right now we're not going to use a rectifier activation function.

101

We're not going to use the release function.

102

We're going to use a leaky reglue.

103

So what I'm going to do now I'm going to go to Google

104

here is Google.

105

And here is the leaky really leaky Leakey who is very close to the loo.

106

But besides having the max of zero and X which actually is the formula for the reglue we have this addition

107

of the negative slope multiplied by the men of zero and x.

108

So that's a slight change in the formula.

109

And if you put that into a graph you can see the difference.

110

There really is a horizontal line equal to zero here for the negative values of x and then the straight

111

line y equals x y for the leak.

112

We have this slight change here which gives some negative values for the negative values of x.

113

So that's the slight difference.

114

I thought it was good to show you this and now why do we choose to use a leaky review instead of a clue.

115

Well again that's an artist's job.

116

It comes from experimentation and research basically for the convolutions of the discriminator.

117

It works better with the leaky reglue than a simple really.

118

So I'm getting back to by then.

119

And therefore if you all agree with the leaky Renu Let's apply a leaky really.

120

So to a player like you we take our end in Mudgal and then that and then we will find very easily the

121

like really.

122

Here it is.

123

And we need to put several arguments of course the negative slope and we'll choose not 0 point zero

124

one but 0 point two.

125

Again a value chosen from experimentation and research.

126

So 0.2.

127

And then we have a second argument in place because false but why not set the value of in place to false

128

will set it to true again if we go back to the PI torch stuck documentation by the way yes this is the

129

PI torch documentation that I highly recommend to go through while watching the tutorials or even any

130

time.

131

That's a great documentation it's very clear.

132

And so if we look at this second argument in place well you can see that's an argument that if set to

133

true can actually do the operation in place the default value is false.

134

But we chose to set it to true to benefit from this option.

135

All right.

136

So let's go back to Python.

137

There we are and we are done now with the leaky well.

138

All right.

139

Let's not forget the comma and let's move on to the next module.

140

So the next module is going to be another convolution.

141

Rest assured we'll get many convolutions as we got many inversed convolutions.

142

So we have a long way to go before making the architecture.

143

So let's do it let's add the second convolution and then that comes to the third is.

144

And let's specify the arguments.

145

So now you understand clearly the logic the input of this new convolution is the output of the previous

146

convolution and therefore it is going to be the 64 feature maps.

147

And now we need to choose a now output.

148

So since it's actually the inverse operation as the previous one which was inverse convolution.

149

Well this time we're not going to reduce the number of feature maps each time for each new convolution

150

as you might have guessed we're going to do the exact opposite.

151

We are going to increase.

152

And more specifically multiply by 2 the number of feature maps in each new convolution.

153

Therefore the size of outputs here that is the new number of feature maps we're getting as the output

154

of this convolution is going to be one hundred and twenty years.

155

There we go.

156

I think now you can almost finish it yourself but we'll never know.

157

Please stay with me.

158

Then we're going to choose a size of four for the kernel strive to a padding of one and no bias bicycle's

159

false right.

160

Second convolution done.

161

Perfect.

162

Next step next step.

163

As I just gave you a hint is to apply a batch normalization to batched normalize each of the 128 new

164

feature maps and so that's exactly the same as before.

165

We're going to take our end in Mudgal than dirt and then we're going to apply a batch norm to D.

166

And since we're Bache normalizing the 128 feature maps will be exactly two inputs.

167

One hundred and 28 exactly like before.

168

No new mystery come the next step.

169

All right.

170

Now that's going to be the same we're going to apply a new leaky reglue.

171

So I'm copying this.

172

I am pasting it right here.

173

And good news.

174

We have the exact same arguments.

175

We don't have to replace anything.

176

We choose a slope of 0.2 and in place.

177

Next step the next step.

178

As you might guess and I strongly recommend this game to try to type the next module in the architecture

179

before me the next module is going to be end that and a new convolution of course comes to the parenthesis.

180

And now of course you guess what we're going to input and what we're going to output.

181

We're going to input what was output by the previous convolution which was 128 feature maps.

182

That becomes the new input of this new convolution and for the output Well we increase this time.

183

The number of feature maps multiplied by two.

184

So we're going to get the new output of two hundred and fifty six feature maps are right.

185

And then again we choose a kernel of size 4 as tried to and a padding of one and again no absolutely

186

no bias bicycle's false perfect come up.

187

Next step the next step is to guess what well to apply a budget normalization.

188

So we take our N and module that batch norm to D.

189

And as arguments we are going to input the number of new feature maps that we got and that we want to

190

batch normalize.

191

And since what we've got is 256 new feature maps.

192

Well we're going to input here 256 to batch normalize these new 256 feature maps perfect next step the

193

leaky renew again.

194

So in that leaky reglue there it is.

195

And that's the same I don't know why I'm typing this again.

196

I just need to copy and paste it and there we go.

197

Now another convolution again.

198

So this time let's not miss it and copy this copy.

199

Based So this time we have 256 feature maps for the input and we double it for the output.

200

So we get five hundred and twelve new feature maps in the output of this new convolution and then same

201

kernel size for a stride of two and a pairing of 1 and No bias.

202

Great then our batched on again and copying this.

203

I'm basing it just below.

204

I am not forgetting to replace the 256 previous feature maps by the new 512 new feature maps that we

205

have to batch norm.

206

Then we apply our leaky reglue with the same parameters.

207

So I just need to copy this pasted here.

208

I hope you are doing all this faster than me.

209

All right then almost over.

210

We have one final convolution to add.

211

This is the last one and pasting it here.

212

And so we're going to get now 512 feature maps of the input and now the question is since this is the

213

last convolution what is going to be the output.

214

Please don't answer 1024.

215

We're done with the convolutions here.

216

We stop playing the game of doubling the feature maps.

217

No this time we have to specify the final output.

218

Therefore the discriminator.

219

And so if you understood the role of the discriminator you should be able to guess the output here.

220

Well I'm going to tell you right now it is actually one.

221

One.

222

That's because the discriminator is returning a discriminating number and number between 0 and 1.

223

So that's a simple vector of one dimension containing this number and therefore that's why we have a

224

1 here and then just a little trap you can not really guess that but we're going to keep a curl size

225

of 4 but then we're going to choose a stride of one and a padding of 0 and then that is the same no

226

bias.

227

All right we're almost done.

228

We have one final thing to do for this architecture.

229

If you look at what we did for the generator.

230

Well you can see that after the last inverse convolution.

231

Well we applied an activation function again to break the new narrative and we going to do the same

232

for the discriminator.

233

But this time it's not going to be a hyperbolic tangent.

234

According to you what is it going to be.

235

It's actually a very classic activation function to use at the end of a neural network.

236

It is the sigmoid function and if you went deep into the theory of the deep convolutional Ganns and

237

especially the part related to the discriminator.

238

Well you would understand that the sigmoid function is the best choice because it returns some values

239

between 0 and 1 and for the discriminator we actually want to be between 0 and 1.

240

Why is that.

241

Well the answer of that question is in the name of the discriminator it's for discriminating reason.

242

The principle of the discriminator is that zero corresponds to the rejection of the image and one corresponds

243

to the acceptance of the image.

244

So 0 means we reject the image and one means we accept the image.

245

Therefore the name discriminator and so basically we want to be between 0 and 1 to do some discrimination.

246

And that's why the output is actually a value between 0 and 1.

247

Because then using classic threshold technique well for the values below 0.5 we will consider it as

248

zeros who will reject the image and put the values above 0.5.

249

We will consider it as one and we will accept the image.

250

So that's very important to understand and therefore the sigmoid function that not only break down in

251

the area but also returns a value between 0 and 1 is the optimal choice for the activation function

252

at the level of the output of the discriminator.

253

All right so if you're convinced we're going to get R and then module and then that and then we get

254

our sigmoid activation function.

255

So it's an object of the sigmoid class which will represent the sigmoid activation function itself and

256

we don't have any argument to input.

257

So congratulations you are done with the architecture of the second brain.

258

You have to create for this sequence.

259

So now are DC again had the two brains.

260

Let's make the forward function to propagate the signal inside the second brain.

261

We'll make it in the next tutorial and until then enjoy computer vision.