Original subtitles

1

Hello and welcome to this new tutorial today we're going to start with the big first step of the implementation

2

of our deep convolutional Ganns that consists of defining the generator.

3

That's the big first step.

4

And the second big step will be to define the discriminator then we'll have our brains and then we will

5

train these brains to generate fake images.

6

But today we're going to start with this big first step.

7

Defining the generator and we'll do it in two steps steps first we will make the class that will define

8

the architecture of the neural network of the generator.

9

So we will put in a sequence to different modules that is the different layers of the neural network

10

then we'll make the forward function to propagate the signal inside and will that work.

11

And then once this class is done we'll be able to create the generator itself that is the neural network

12

of the generator by simply creating an object of the class an instance of the class.

13

So let's do this let's make this class.

14

So to define a class in Python We start with class and then we give the name of the class which is going

15

to be G for regenerator and then we'll call the class for the discriminator D.

16

And now we're going to use a trick an object oriented programming language which is inheritance.

17

We are going to inherit from in that module which contains all the tools that allow us to build a neural

18

network and it's called an end or modules because the modules are the different applications and connections

19

you can make inside a neural network.

20

For example one module can be a convolution.

21

Another module can be a full connection.

22

Therefore the name and to apply the inheritance there is nothing more simple we just need to put as

23

argument of this class G.

24

The module we want to inherit from which is and that module.

25

There we go this one.

26

And in that module so that introduces a new class.

27

And now inside this new class we have to define the whole architecture.

28

And besides we'll include the forward function to be able to afford propagate a signal.

29

That's the advantage of making a class A class is an advance structure inside of which you can include

30

some properties of the future instances of the class that will be created.

31

That is the future object.

32

And also some different functions that will be like some tools of your object.

33

Quick and the best example for those of you who are seeing your class for the first time.

34

Well we could define a self-driving car with a class in one class we could include all the properties

35

of the Sandrine car and then different functions for example to move forward function to turn the function

36

to turn right function the stop function.

37

All these functions that would allow to control the self-driving car.

38

Well here that's exactly the same we're making in class to not only define the architecture of the neural

39

network of the generator but also we'll add the forward function which is like a tool that we can apply

40

on the neural network.

41

And this tool will be used to propagate the signal inside the neural network.

42

So let's do this let's first start by defining the architecture of the neural network.

43

And as for every neural network that we make with a class we do this inside that in its function the

44

innate function is the function you always start with a class.

45

It is the function that defines the properties of the future objects that will be created from your

46

class.

47

And speaking of this object that's exactly the argument that we have to employ right now.

48

And this is called self self has been a mystery for a lot of people but there is absolutely no mystery

49

about it.

50

It just refers to the future object that will be created.

51

It's just a common way to specify that when we create a variable we use the self to specify that this

52

variable belongs to the object.

53

So each time I put self.

54

That means I am referring to the object.

55

That means I'm taking the object and setting some properties to it.

56

So self.

57

And now we are arguing to activate the inheritance because we import the n and module to inherit from

58

it.

59

But now we have to activate this inheritance and to do this we use the super function inside of which

60

we input our Class G which is inheriting from the end in that module right here.

61

And then the second argument is self because we will use the tools of that module on our object because

62

our object is nothing else than the new one that work at the generator and the neural network and the

63

generator will be composed of the different modules.

64

And in that module the conclusions the full connections and more.

65

All right and then that's not over we just need to add a that and underscore underscore in it double

66

underscore again and parenthesis.

67

All right.

68

Nothing more important to know about this.

69

That's just what we have to type here to activate the inheritance of the end and that much our next

70

step.

71

Now the next step is to make a meta module what is a metal module.

72

That's a huge module that will be composed of several modules.

73

I remind that by module I mean the different layers the different connections inside the neural network.

74

So now we'll create a big metal module which will be a sequence of several modules some convolutions

75

full connections inversed convolutions and more integrate this metal module.

76

Well we're going to take our object because this metal module is going to be a property of the object

77

and therefore I'm taking my object and this metal module of the object will be called Maine.

78

Main will be the metal module that will contain all the different modules in a sequence of layers.

79

And speaking of sequence of layers.

80

That's exactly what we'll introduce right now with the N-N that sequence show class inside of which

81

will create the different modules.

82

All right.

83

So you probably recognize that sequential is actually a class so self that main is actually a new object

84

of the sequential class but of course this object represents this method module composed of the different

85

modules.

86

All right so now we're starting to define the architecture of the neural network and let's start with

87

the first module.

88

According to you according to the intuition lectures you saw with curial what is going to be this first

89

module.

90

Well of course that's going to be a non-first convolution not a convolution.

91

And in verse 1 that is exactly the inverse of a convolution.

92

And why do we have to start with an inverse convolution.

93

Well that's because the role of the generator is to generate some fake images.

94

And therefore since CNN takes as input some images and returns as output a vector.

95

Well and then CNN will do exactly the opposite.

96

It will take as inputs a vector we will create that vector we'll call it noise later on in the training

97

that will be the input of this inverse CNN and it will return a fake image.

98

So you're going to see that the input of this inverse CNN that we're about to make will be a random

99

vector of size 100.

100

And to get the inverse of a convolution in PI torch we're going to use the can transpose to the class

101

that is that this inverse convolution module that we're going to create will be an object of the current

102

transpose to the class.

103

So let's do it.

104

We take our end in that module because this can transpose to the class is taken from the end module

105

so end then dot com capitals C transpose to the.

106

Here is this one.

107

That's the inverse convolution.

108

The first module of our big Maira module that is the first module of our neural network can be transposed

109

to D.

110

And now we need to input several arguments.

111

Get ready for them as you can see.

112

There are many of them.

113

All right.

114

So the first argument is 100.

115

That's the size of the input 100 meaning that the inputs are inverse CNN will be a vector of size 100.

116

Then the second argument is 512.

117

And that's the number of feature maps of the output.

118

Then the third argument is four.

119

And that's the size of the kernel which means that the kernels will be squares of size four by four

120

then the next argument is one.

121

And that's the stride and the last argument is zero and that's the padding.

122

By the way the architecture that I'm about to define right now I didn't invent it.

123

It is coming from a lot of experimenting that was done by researchers and the machine learning and AI

124

contributors.

125

So I did not invent it.

126

I just took an architecture that is open source and that turns out to work very well for adversarial

127

networks.

128

So don't worry about the choice of the numbers.

129

They just come from experimentation.

130

All right and then we have one last argument and I'm specifying it because it is actually about the

131

bias which by default is equal to true.

132

But I'm going to set it equal to false because we don't want any bias.

133

It works better this way.

134

And therefore I'm saying the bias here to false.

135

So one hundred for the size of the input 512 feature maps and the output meaning the output of the inverse

136

convolution not the final output.

137

Then a kernel of size for four by four is stride of 1 a batting of zero and no buyers.

138

That's the first Mudgal of our neural network.

139

Congratulations.

140

You just applied an inversed convolution all right and now we are going to normalize all the features

141

along the dimension of the Bachche the diamonds and of the batch is 512.

142

We have 512 feature maps and we're going to Bachan on each of these 512 feature maps and to do this

143

we take again our and then Mudgal and then we apply the batch we should find in here.

144

Here we go Bachan on 2D and as arguments we need to input the number of feature maps we want to bet

145

on.

146

And since the 512 feature maps Well we are going to bet on 512 feature maps.

147

All right.

148

So playing batched normalization is pretty simple.

149

Then we are going to apply Ereli to rectification for the non-linearity of the neural network that is

150

to break the linearity and to do this.

151

Nothing more simple things to our end and module we take are in a module that and we apply the review

152

rectifier activation function and as arguments we just need to input just like that.

153

That applies really to rectification then we are going to apply another inverse convolution.

154

So I'm simply going to take my end and can transpose to the module.

155

Pasting it here.

156

And then here we go with the series of arguments.

157

So this time we don't have 100 for the size of input because the input is now the output of the previous

158

module which was an inverse convolution but in the output we got 512 feature maps and these feature

159

maps become the new input of our new inversed conclusion.

160

So this time we don't input 100 year when put 512.

161

That's the number of inputs of this new inverse convolution.

162

Then we need to choose a new number feature maps in the output of this new inverse convolution.

163

And this time we're not going to pick five phonons well.

164

We're going to pick two hundred and fifty six.

165

Again that's a choice motivated by experimentation.

166

That's really not his job to figure out such an architecture.

167

All right then we're going to choose a kernel size of four four by four then as I tried to this time

168

and padding of one and then same we don't want any buyers and therefore adding Byars equals false perfect.

169

That's our next module.

170

We now have two inversed convolution.

171

Our generator is getting into shape.

172

So now we're going to do the same we're going to batched norm each of the new 256 feature maps.

173

So I'm just going to take this and that batch Enorme 2D it here but then replace 512 by 256 because

174

we're batch norming each of the new 256 feature maps and then we apply another rectification which is

175

going to be the loop again.

176

So here let's just copy this and paste it here.

177

All right then guess what we're going to do again.

178

We're going to apply another inverse convolution.

179

So I am taking this.

180

And then that can transpose to D.

181

I am pasting it here.

182

And then the input is actually the output of the previous operation that is the inverse convolution.

183

So that's 256 the 256 outputs feature maps become the input of our new inversed convolution.

184

So 256.

185

And now guess what we want in the outputs.

186

Well you probably understood the logic we wanted divided by two.

187

So we're going to take one hundred and twenty eight output future maps.

188

In this new inverse convolution and then the good news is that we're keeping a size four by four for

189

the kernels.

190

Let's try to add a padding of one and then no bias.

191

Congratulations.

192

You just applied your third inverse convolution.

193

And so then that's the same we're going to apply a batch known to the two batched norm.

194

Each of the new 128 feature maps.

195

So here I am replacing tournament 56 by 128 and we are going to apply reglue rectification to make sure

196

we break the linearity for the non-linearity of the neural network.

197

All right.

198

And then guess what we're going to do again.

199

We are going to apply an inversed convolution again.

200

So I'm taking this again I'm taking the cones transpose to 2D and I'm pasting in here and this time

201

we have as input of this new inverse convolution.

202

One hundred and twenty eight feature maps because that was the output of the previous inverse convolution.

203

And this time guess what we want for the number of the output feature maps of this new inverse convolution.

204

Well of course we want 64 new feature maps and then good news again we're keeping a size of four by

205

four for the Colonel's a stride of two and a betting of one.

206

And then nobody has a right.

207

We're getting close to the end.

208

We need again to apply a batch norm.

209

So let's copy this again.

210

We are going to batched norm each of the new 64 feature maps that are the output of our new inverse

211

convolution.

212

And then we are going to apply a really rectification to again break the linearity.

213

Right there we go almost over.

214

Don't get crazy too fast.

215

We have one final inverse convolution to make.

216

So that's exactly what we're going to do.

217

We are going to apply our last module the inverse convolution and I'm just copying the name of the module

218

because this time we are going to input different arguments.

219

So the first argument is the number of inputs which is the output of the previous inverse convolution.

220

So that's 64 again.

221

And then for the output That's important since we're making the generator that is supposed to generate

222

some fake images.

223

And since these fake images are going to be with three channels while the output of the generator is

224

exactly going to be the three channels of the fake images and therefore the number of dimensions for

225

the output is going to be three corresponding to the three channels.

226

All right.

227

And then actually that's the same.

228

We're going to use kernels of size four by four a stride of two the padding of one.

229

And again no bias.

230

So I can just copy this and paste it here and there we go.

231

We have applied all our inversed convolution that's almost over.

232

We just need to end this up with a hyperbolic tangent rectification to not only break the law in the

233

area again but also to make sure we are between minus 1 and plus 1.

234

Have a look at the age function you'll see that the values are between minus 1 and plus 1.

235

And also it is centered around 0.

236

And why do we want to output values to fall between minus 1 and plus 1 and being centered around 0.

237

That's because we want the same standards as the images of the data set.

238

And because the created images of the generator will become the input of the discriminator that will

239

come after that.

240

All right so now we just need to and this architecture with our n module again and then that we apply

241

the term age to hyperbolic tangent rectification for the non-linearity of the neural network and to

242

be between minus 1 and plus 1 centered around zero.

243

Whew.

244

Congratulations.

245

You made a neural network.

246

That was definitely not a piece of cake.

247

So I guess now it would be nice to have a break in the next story will make the food function that will

248

propagate the signal that is the input of the generator through all the different layers of this new

249

will that work.

250

So have a good break and I'll see you in the next tutorial.

251

Until then enjoy the revision.