Original subtitles

1

Hello welcome back to the course on computer vision we talk about the horror like features the name

2

Haar comes from Alfred Haar who was a Hungarian mathematician in the 19th century and he developed a

3

mathematical concept called the haar wavelet which is a basis of way for functions which act similar

4

to the four year transformation but we're not going to get into detail on that right now because we

5

don't really need that we just need the horror like features which have since been derived from the

6

hard wavelet and our that's why they called Horo like features they just kind of like the descendants

7

of the haar wavelet and so we're going to look at the HARLICK features straight away.

8

So what are these features.

9

They are.

10

Let's first look at them.

11

They are the edge features the line features and the four rectangle features these are the Horlick features

12

that Viola Jones male and Jones identified in their research in their paper.

13

And so what are they.

14

What does it mean to be a feature.

15

Basically what we have is pixels.

16

So on an image you might have somewhere for example like a edge.

17

Right so an edge of feature like for instance an edge of an image for example just imagine like a photo

18

of a table or something.

19

Then part of the table where you can see the table top might be light and or lighter than the part where

20

the table ends might be darker because it's further away it's the floor or the carpet might be a different

21

color or something.

22

And so you can identify that that's the end of the table because you know there's a line of pixels which

23

are brighter line of pixels which are dark in terms of the age in terms of line features you might you

24

know might identify some objects and it doesn't have to be just like one line of pixel this could be

25

like three columns of pixels and then you know 15 rows of pixels.

26

So these are scalable.

27

They can be longer or they can be wider and so on.

28

So let's have a look at how all of these work out in the in terms of a face.

29

Right.

30

So here we've got our friend J.D. and let's see if we can identify some Horlick features here and that

31

will help us understand better why exactly they were picked.

32

What's a role what purpose they serve.

33

So have a look here.

34

So if you look at his mouth you can see here there is a tiny tiny tiny blackline then.

35

So that line that part is darker than the lips.

36

So that's a horror like feature.

37

You can see that that's a feature that's a line feature because that part is darker than that.

38

Those two parts bottom and top.

39

And this is all very rough very very very kind of rough approach to this image but nevertheless it works.

40

And you will see why it works.

41

Further down as we talk about things like the integral image and so on.

42

But for now let's just let's just stick to this bar so we kind of like breaking out these features we're

43

trying to see them in this image.

44

So what else can we see here.

45

Can you see any other of the features that we discussed.

46

If you if we go back we've got the edge features.

47

So we just identified this one we just identified a line feature.

48

Can we find maybe an age feature in several There's a line feature they'll look at the eyebrow you can

49

see that the eyebrow is darker than the forehead so let's put a feature there so you can see that that

50

signifies the edge of the forehead that line that eyebrow.

51

And so therefore the hard feature of an edge can serve a as a concern to show where this break this

52

division happens.

53

And that's normally going to be the case for in most photos that you process.

54

Most of the time the eyebrow is going to be darker in the face.

55

So for you know some some rare instances or some non frequent instances in most cases are going to be

56

like this and we'll I will talk about how all of these features were together.

57

So it doesn't have to be like every single time it's is like this but they're going to have a kind of

58

like a voting system that the more features the more likely it is a face.

59

Let's have a look at some more.

60

What can we see on for example on the nose.

61

Can we see that part of the nose is brighter than the other part.

62

So you can see here.

63

Let's see on the left here.

64

The nose is brighter.

65

The bridge of the nose is brighter than this part.

66

And that's normally going to be the case.

67

So if you look at pretty much any photo in grayscale you will often find that the bridge of the nose

68

is brighter than either just one side or even both sides.

69

It depends on the lighting.

70

Sometimes you might find here like a line type of feature so this part is wide and then you've got dark

71

and dark.

72

That's just how the human face reflects light in this case the lights coming from the left.

73

And that's why the left part is a bit brighter overall.

74

The right parts are.

75

But nevertheless you can see this edge.

76

So here you are nobody have a line or an edge feature.

77

That's exactly what we're drawing here.

78

There's our edge feature.

79

And then let's have a look another one.

80

You can see the eye for instance because though the outside outside of the outer part of the eye is

81

white and the inner part is not white.

82

It means it's dark or so it doesn't have to be like black and white it's black and white would be ideal.

83

So for instance the eyebrow in the forehead in this case are very very close to black and white and

84

because of the eye it's less close.

85

But nevertheless it still works you can see that you can identify that feature inside the eye as you

86

could could go keep going like that you could find another one for the other eye than eyebrow.

87

You could you just keep finding these features and some of them would be dependent on the lighting some

88

of them would be dependent on the person inside.

89

But overall you'd have kind of like a set of features that are predominantly present on most human faces

90

or like you have like a set of features most of which are present on any face meaning that if you you

91

have to have like at least 80 or 90 percent of the features like speaking roughly of course we'll get

92

into more detail about this further down.

93

But you'd have a speaker after you'd have to have like 80 or 90 percent of those features actually and

94

fight for it something to be a face would be very rare that you would have a face that would have only

95

half of those.

96

So now let's see how exactly this works in terms of the pixels.

97

So let's talk about the nose bridge featured or identified this age feature over here.

98

So if we zoom in a little bit you can see it is a feature.

99

Now I'm going to draw.

100

I'm going to show the nose without the feature overlaid but instead we're going to have the pixels so

101

of course this is a very simplified representation because there are many more pixels in this space.

102

But just for argument's sake let's say that there's four pixels by eight pixels thirty two pixels in

103

total and this nose.

104

And now we're going to understand how the algorithm works.

105

So let's ride out these pixels and here I've given them values between 0 and 10 there is 0 and 1 0 means

106

very white.

107

So 100 percent white 10 mean and 1 means 100 percent black and anywhere in between is greyscale.

108

And of course greyscale is from 0 to 255 because it's like 256 values and so on.

109

But we're not going to bother with writing them a lot.

110

We're just going to normalize them two from 0 to 1 and just keep it that way.

111

So it will be easy just for us to discuss.

112

And so here you can look at this image and you can see that this is not 100 percent white.

113

This would be 100 percent white.

114

But this is close to white.

115

So in an image like for argument's sake again this could be like 0.1 this pixel could have like a 0.1

116

intensity given that zeros and white and one is 100 percent like this pixel is a bit darker.

117

So is there a point to this pixel is about the same brightness or upon one's report ones or report one

118

0.2 it's a bit darker over here.

119

0.3 it's even darker if you compare to that one.

120

This one is 0.1.

121

This is a point and so on and so on so you can see that these are they're not all zeros but they're

122

close.

123

They're on the side you can see that this one is about 0.4 or somewhere on their 0.5.

124

This gets darker 0.8 3.6 this was a bit brighter sort of potent force similar to that.

125

Closer to that than to that and then to that for instance the reported fires are pretty high.

126

So you can see kind of the values here.

127

Now what's the algorithm will do what the algorithm will do is now that we've identified all of this

128

so this is this is where we see the feature we can see that kind of like white and black but this is

129

what is happening in reality not actually 100 percent black and Harmsen white that somewhere greyscale.

130

So what the Belgians algorithm will do is it will compare.

131

How close is this real world tonight or what is happening in the real world to the ideal Haar like feature

132

that it's looking for the white and black.

133

How close is it.

134

So how would it do that.

135

Well it will take the average of these Vaso First of all it will calculate the sum.

136

That's the first thing it does it calculates the sum of all these pixels and then divide by the number

137

of pixels it takes the average intensity of the white pixels and 0.1 6:06 takes the same thing for the

138

black pixels takes the sum of all of these values and now takes the average.

139

So as devised by whatever 16 here adds them up to as 16 gets average and this is the average intensity

140

of all of these pixels Fizer 0.06.

141

And now it's simply subtract one from the other subtracts this from that.

142

So the black minus the white is 0.4 02.

143

So these pixels and minus these pixels are the average minus averages 0.4.

144

Now so the question is what would this difference be in the ideal scenario.

145

Kind of the ideal scenario that where you would find exactly this feature where this would be all black

146

and this will be all white.

147

Well exactly right.

148

If it is all black this is all one.

149

If so why this is all zeros.

150

So it'd be one minus zero the average be one the average here would be 0 1 0 would be 1.

151

So basically in this in the in this approach the closer it is to one the closer the real world scenario.

152

This one is to what is being looked for.

153

Now of course we understand that we're humans we're not robots we don't have black and white delimited

154

parts like that so when we're And you know it depends on the lighting depends on on all these different

155

things.

156

So of course you're never going to have an exact one here or an exact zero.

157

This is always going to be some sort of gradients some sort of different colors shades and so on.

158

That's why the bell Jones has some thresholds and through the training which we'll discuss further down

159

through the training these thresholds identified.

160

And for instance for this specific feature in this position it's identified that the minimum threshold

161

might be 0.3.

162

And so whenever you get a result a calculated value here that is greater than 0.3 that means that indeed

163

this is somewhat similar to this HARLICK feature which we're looking for and we're going to say that

164

yes this Harlech feature is present in the image where in that position where we were looking for it

165

if on the other hand you get like 0.1 or basically a value that's less than your threshold then is the

166

difference between the two colorings for instance here.

167

For answers if you just basically applied looked for this feature because the algorithm doesn't know

168

that it needs to look for it here it will just look for it everywhere and look for it here or look for

169

a hero.

170

It's looking for the nose and at the end all it knows about the nose is that the nose looks like this

171

like this like this feature.

172

So it's going to look for it everywhere.

173

So if for instance algorithm were to apply this exact HARLICK feature some over here in the top left

174

corner then the difference between the real world of whites and the real or black would be nothing right

175

because it would be zero.

176

In this case because it's exactly the same it's just looking at the wall in the white and the wall in

177

the black.

178

So it would look for the nose in many different places and then it might find a nose somewhere else.

179

But then through other features it will understand that that's not the face.

180

But for now what will it find here in.

181

Once it gets to the actual nose it will see that yes.

182

In this case it it surpasses the three threshold that exceeds the threshold so this we're going to say

183

that there is this potentially or knows over here that might be a potential nose over there that might

184

be a potential nose or here but it's definitely going to say that there is no nose in many other places

185

and then through the other features and we'll learn more about this throughout the section.

186

It will be able to identify where exactly the nose is where exactly the faces but that's that's the

187

starting point.

188

And so that's where the threshold comes in.

189

The threshold is very useful to understand if this feature is actually present or if this is or we should

190

move on and there is this feature is not present in that location we're looking.

191

So that's how the Horlick features work.

192

Very important.

193

They lie in the foundation of this algorithm.

194

They're kind of like once those features that were identified in the face the like for eyebrows and

195

so on.

196

Once you once the Aluf you don't have to do this by your algorithm identifies through training a a number

197

of features quite a large number of features then that is kind of like forms the basis for to understand

198

what elements roughly a face is constructed from and then it will be able to use those to detect the

199

face.

200

But in the in a nutshell this is how it works through the pixels through the white and black regions

201

which it calculates averages for and then subtract.

202

So there we go.

203

Hope you enjoyed today said Tauriel and I'll look for see in the next one.

204

And until then enjoy computer vision.