Original subtitles

1

Hello and welcome actually a course on computer vision.

2

And today we're talking about the multi box concept behind the as the algorithm.

3

This is going to be quite a visual tutorial so there we have Avalon standing on a pontoon at Lake Bonynge

4

in Slovenia just as a side note if you're ever in Sylvania or around there definitely visit lake behind.

5

It's a beautiful magical place as you can see it's it's amazing there.

6

There's also those the Alps behind you is really great to have a lookout tower there.

7

You can actually go up and see all that like a huge part of Alps from up there.

8

Very lovely We were there during our European road trip when we are visiting students all across Europe.

9

And is there any great stuff.

10

They have two big lakes and Savannah Lake Bled and Lake voyage and this is like Boeing's bigger one.

11

It's a bit further away from the main cities but it is worth the trip.

12

And so as you can see we're very guilty posture on identifying him as a person that he's standing there

13

pointing.

14

And what we're going to do today is we're going to take this pork's as the ground troops and then we're

15

going to see how the the Allegan works with its own boxes to understand to actually identify ODLAND

16

and build this same exact box.

17

And first things first.

18

What is the ground truth.

19

Well a ground truth is a concept that's not just using computers is used in many places and it separates

20

the observed evidence or empirical evidence from inferred evidence.

21

So for instance when you go and actually look at something and you see something and you can physically

22

understand that this is a true state of things that's your ground truth there.

23

There is no two ways about it.

24

No there's no gray areas as you can see exactly that this is the box that should be wrong.

25

Whereas when you apply an algorithm and it goes processes all of this data through its convolutional

26

neural networks and so on and comes back with boxes those are not going to be ground truth boxes those

27

are going to be inferred boxes those are going to be suggestion brought up by our group even if they're

28

correct they're still merely suggestions and in order to make sure that they're correct we need to have

29

the ground truth.

30

And that's why in the process of training these networks the D-Fla. in particular we need the ground

31

true.

32

So we need these boxes to be identified we cannot just train the algorithm.

33

If these boxes are not present in the first place that's something important to keep in mind for that

34

training process.

35

And yeah so there is a lull.

36

Let's see how the algorithm will work and build its boxes and how the training will happen.

37

So what this is they will do is will break down the image into segments and then for every segment it

38

will construct several boxes will construct the box like that looks like that and a box like that for

39

for instance so these exact exact set up can vary.

40

But we're going to stick to this one this is the most generic one and basically the whole image will

41

get covered with these boxes.

42

As you can see it's covered entirely.

43

And then for every single one of these boxes is going to ask the question is there an object in there

44

or not and not just an object is to ask that question every single class of objects that it's it's trained

45

for or in the training processes that is going to ask that question for every single class of all that

46

it is training for.

47

And so basically in this case it will identify that none of these boxes except for the ones highlighted

48

in red actually have an object of interest and then we're going to assume that in this case for simplicity's

49

sake they we're only looking for people.

50

There are people in the background there but they're very small.

51

And the algorithm will not be able to pick up their features and therefore will not detect those people.

52

I will talk about size and scale further down and inch and others Tauriel in the section for now which

53

is going to focus in on Lot and we can see that some of the boxes have picked up some features of a

54

human being in that current form.

55

So if we remove all that let's just look at one box just so that it's clear why for instance this box

56

is able to pick up that there is a person within this box or it doesn't have to be a full person.

57

This is what I wanted to point out here doesn't have to be a fool personal object as long as it can

58

find a sufficient number of features it will produce a probability it will say that with a likelihood

59

of 80 percent or 70 percent.

60

What I have in this box is the human being or as a car or as a both or as a cat or a dog.

61

So in this case it sees enough features to say that there is a human or a person inside this box.

62

And then all of these ones that we have identified here can see the same almost the same features and

63

they're very different features but nevertheless they all have enough identify each with a green one

64

you can see that that green one for instance.

65

It can it only sees from the elbow downwards.

66

So just for the sake of this tutorial I decided to assume that it's not seeing enough features although

67

as you'll see from the practical side of things that is actually that would be enough features for the

68

S's algorithm to detect a human being.

69

But just for our case I just wanted to show you that not always these three boxes like you can see that

70

the boxes come in threes.

71

In our case that does not always tally.

72

So if one of them takes a person that doesn't mean all the time.

73

So in this case that would be draw.

74

And so far we have not six but five officers that have to take that approach.

75

And so that is the ideal case and that's what we're striving towards.

76

That's what we want to train the next hope to do.

77

And the way it will happen is here's our network.

78

And every time all of these boxes predict something we'll have an output so I'll bring them out here

79

so for instance they'll look at these small areas.

80

For now we'll talk about them later.

81

We're dealing with this layer over here and so we've got all these boxes and they all predict something.

82

And so then the on neural network searches for features that predicts that for instance in these boxes

83

in these boxes that there's a human that's what we want.

84

But for example if it had predicted that there is in these boxes as a human or in these boxes a human

85

and then on the other hand it predicted that some of these boxes don't have a human at the end would

86

have an error.

87

How would we have this error because we actually have the ground truth.

88

We have the ground truth with this white box and we would actually know by overlaying these images the

89

algorithm knows exactly which boxes should say yes there is a human.

90

And which boxes should say no.

91

So it really knows the way what we did actually here is with the red boxes are the ones that I ideally

92

should say there's a human and we want them to say human and all other boxes which we've deleted should

93

say there is no human there is no person.

94

And so unless that is the case there will be an error.

95

So if say for instance a box or here says that there is a human and a box or a says that there is a

96

human and one or a couple of these boxes don't pick up the humans there'll be an error.

97

And at the end of the algorithm that error can be calculated and can be expressed mathematically we

98

don't get into that that you can find more information on that in the annex on convolutional neural

99

networks.

100

But basically that area can be expressed and then what will happen with that error is will be back propagated

101

through the network as is just a visual representation of the to this point just back propagate through

102

a network and the weights of the network will be updated in order to better accommodate that so that

103

next time this box won't predict a human and this box will predict a human.

104

But these boxes will predict the.

105

So it takes many iterations is going to happen on and on and on and on but that's how the training process

106

goes.

107

And so the important thing here to remember is that we unlike just simple convolutional neural networks

108

where we just want to predict if there's a dog on the if there's not a dog in this case we actually

109

need to find where on the image there is a person and so that we need a ground truth.

110

So it's not sufficient for us as for this image itself to be labeled that there is a person or there

111

is not a person.

112

No that's not enough.

113

We have to know exactly where the person is so we need this box already on the image for the training

114

process.

115

And the way to think about it kind of a very intuitive way of thinking about it is take each one of

116

these boxes as a separate.

117

They think of each one of these boxes as a separate example of the convolutional neural network.

118

In practice if you think of each one of these boxes as a separate image that's one.

119

If you take each one of these boxes as a separate image then yes then you that that kind of is very

120

similar to a convolutional in your own codes because for each one of the boxes you just need to make

121

sure it predicts that there is a human inside this box.

122

And you also need to know for the purpose of this that indeed there was a human side as well.

123

So that's the ground truth in that case.

124

So that's how it works.

125

Like basically all of those boxes that we saw around that were built plotted around the image.

126

You just can think of them as separate images in their own right.

127

And each one is trying to understand is there a human inside my image or is it or not it's human an

128

image and doesn't care about the rest.

129

But then the algorithm puts everything together on the hands and it takes the aggregate error propagates

130

and network and trains everything up like that.

131

So there we go that's the multi box concept of the essence the algorithm.

132

And we will continue that etc..

133

I look forward to seeing you there.

134

And until then enjoy computer vision.