Original subtitles

1

Hello and welcome back to the course on computer vision in today's tutorial we're talking about the

2

integral image.

3

So what is this image that is so important in computer vision in the way all the Jones algorithm.

4

Well previously we stopped off here where we introduced the horror like features and we found out how

5

they were calculated so briefly to recap we have a certain feature that is commonly found in certain

6

parts of the face for instance here we can see this line where it is like more white half was more black

7

is Representative sometimes on some certain photos in some sort in a certain lighting it can be present

8

in the nose here.

9

And then we calculated that through the actual pixel values in the image.

10

When we talked about threshold.

11

So basically we found out that in order to calculate this in value for the feature to find out if it's

12

present in the image or not so that the criterion for that to evaluate the criterion for us and feature

13

we need to make perform calculations and we need to calculate the sum of all of these values and this

14

rectangle and the sum of all the values in this rectangle.

15

Strike one from the other.

16

And as you can imagine this can be quite a costly exercise in terms of computations.

17

It take quite some time to add all these numbers up especially if you have lots of these features at

18

evaluating and especially if these features are quite large.

19

So there is a hack to doing this very quickly and this is where the integral image comes in.

20

So let's just imagine that we have an image with a certain number of pixels us as shown here.

21

And for simplicity's sake we're only going to look at the intensity of the gray scale from 0 to 1.

22

We're just going to do it from 0 to 10 so we'll just multiply by time just so it's easier for us to

23

look at this example we don't have to deal in decimal point so let's randomly fill in this image.

24

Doesn't really matter what kind of image we're talking about.

25

Any image can be represented in this type of form where we have a value for every single pixel.

26

And so let's say we're calculating a whore like feature and we need to in order to calculate evaluate

27

that feature on this image we need to calculate the sum of all of the intensities in this specific valks

28

that we've outlined.

29

So what we normally do is we would just take 10 plus four plus nine plus eight percent plus four plus

30

one plus nine point nine percent plus we'll add them up and we would get some.

31

As we discussed this can become potentially expensive if you're looking at large features like this

32

one and there a three by four that's 12 values that you have to add if you if they can grow if they're

33

larger in size then is it going to be even more values that you need.

34

So basically the the larger the feature The larger the image the more you have to do some calculations

35

the more expensive that calculation is in terms of time and as we know in computer vision if we especially

36

for real time computer vision then time is very valuable we cannot.

37

We want to minimize the amount of time we spend calculating and plus you need to not only evaluate one

38

feature there can be lots and lots of features that you want to evaluate in one image.

39

So again another reason to minimize the time spent on this.

40

So what are going to do now is we are going to put this image the side here.

41

Are we going to forget about the box for now.

42

We'll get back to it.

43

I now we're going to move to constructing the integral image So here we've got the image on the left

44

and on the right.

45

We're going to create the integral image integral image is exactly the same size as the original image.

46

And all you need to do to calculate the integral image is a very interesting operation.

47

So for any given square in the interval and let's take the square for example the value that will reside

48

in that square is the sum of the values on the original image above and to the left.

49

So the square is here.

50

That means we take all of the values that are sitting here and the original image and we add them put

51

their sum into the integral image.

52

So in this case it's one plus 9 10 2 plus 8 25 plus 0 25.

53

And that's why we're going to put 25 again.

54

Now there's another one just to solidify this process was just around the square here.

55

So in order to calculate this square we're going to have to calculate all of the values they like to

56

take.

57

And it's not like over here which is this square and then go up and left and everything in this rectangle.

58

We need to add them up and put them put them into that squares.

59

If you add them up you'll get all 134.

60

And that's the value that goes into this square in integral image.

61

Now what we're going to do is we're going to fill in the whole integral image exactly that way so you

62

can just if you would like to you can pull this video and you just stick your hand value.

63

So for instance 201 here would be the sum of all of the values that are in this rectangle.

64

And so that is how you calculate the integral image and so why do why do we use integral.

65

Because it's a very efficient hack to the exact problem that we have with the exact challenge we have

66

of calculating those howre like features.

67

And the reason for that is because HARLICK features are actually rectangle so let's have a look at an

68

example here.

69

So back to square that we originally were rectangle there were originally identified so we wanted we

70

said that hypothetically we want to calculate the sum of all of the values inside this rectangle in

71

order to evaluate a certain horror like feature.

72

And so what we would do here is we would use the integral image to help us remember we said that in

73

this original image if if we took our original approach of just adding stuff up we would have to do

74

four times three 12 you'd have to add up 12 values.

75

Now let's see one integral image can help us do so in an integral image what we'll do is we'll take

76

the value that represents this bottom right corner but we'll take an integral image.

77

So there it is.

78

And so this value is actually what does it represent.

79

It represents the sum of all of the values in here.

80

So let's outline them.

81

There they are.

82

So that's the sum of all those values is 235 So write that down at the bottom 235.

83

Now we're going to take the value in the top right corner just above the top right corner of rectangle.

84

And that value that 83 is the sum of everything.

85

Above all rectangle above.

86

In this area in these first two lines.

87

So if we subtract this new value from our existing value from 2 3 5 then we will remove all of these

88

values.

89

So we'll take it out.

90

So you can see like 235 we've got this big red box that's shaded.

91

If we subtract eighty three we will get just this box.

92

So let's write that down.

93

Now we're going to take the value in the top left just above the top left corner of rectangle So this

94

value over here of course ponse to this is a space in the image or here.

95

And that means if we add it we will add back this rectangle.

96

So it might be a bit counterintuitive but we're going to add it back because that's going to help us

97

in the next step.

98

So we added back and now finally we're going to take the value at the bottom left corner over here and

99

we'll subtract that as well because that represents the values on all of this rectangle.

100

So if we subtract that and write that down as you can see what we're left with is exactly that rectangle

101

that we are after in the first place.

102

So we took that value subtract that value.

103

So it took that value subtract that value took the green bottom right value over here.

104

Thirty five to check that A-3 added 47 to 71 34.

105

And the result is 65 and that is exactly what we were after.

106

So as you can see we only needed to perform four operations in order to calculate this rectangle the

107

sum of the values inside this rectangle as opposed to.

108

So when we need to to add or subtract for values as opposed to 12 values if you're doing it by hand

109

if we had to calculate every single you know add them all up here.

110

And the beauty of this is you will always only need to look at for values regardless of the size of

111

the rectangle so even if your rectangle is a thousand by a thousand you will still only need to look

112

at four values in the integral image whereas in the original image you would have to deal with 1000

113

times 1000 You have to deal with a million values.

114

So as you can see the integral image doesn't scale it does the time it takes you to process the integral

115

image doesn't change proportionately and doesn't change at all regardless of the size of the feature

116

that you're dealing with.

117

And therefore it makes it very easy and once again it is only possible to use an integral image because

118

as you remember all of the horror like features there are actually rectangles there actually are composed

119

comprised of rectangles black or white rectangle so you always will need to look at rectangles as like

120

a circle in there if there was some odd looking shape then it wouldn't be possible to help to use the

121

integral image.

122

And so that's why the hard features of while powerful are because even though they might not be ideal

123

they might not ideally find the right of the exact features of an image there might be a bit rigid some

124

in some places because they're just rectangles at the same time they make up for that with their computational

125

efficiency because we're using the integral image and that's one of the major advantages of horror like

126

features because you might find better features like circles or some other features that might be better

127

classifiers overall but because hard but HARLICK features they are much easier to calculate through

128

the integral image therefore you can use a lot of them you can use it you can evaluate them much much

129

quicker and that's why they make up.

130

So there we go that's how the integral image works.

131

And that's why it's such a huge advantage for the viola Jones algorithm that's why it implies it.

132

And yeah I hope you enjoyed it as a tural and aliquot seeing it next time.

133

And until then enjoy computer vision.