Original subtitles

Instructor: Hi, and welcome back.

Let's begin this lesson by introducing binary encoding.

We will start from the

ordinal numbers we assigned earlier.

Bread is represented by the number one,

yogurt by the number two,

and muffin is designated with three.

Binary encoding implies we should turn

these numbers into binary.

One in binary is zero one,

so bread would be zero one.

Two in binary is one zero.

So, yogurt would be one zero.

Three in binary is one one.

So, muffins would be one one.

The next step of the process is to divide

these into different columns,

as if we were creating two new variables.

For the first one, bread is zero,

yogurt is one and muffins are one.

For the second variable, bread is one,

yogurt is zero, and muffins are one.

We have differentiated between the three categories

and have removed the order.

However, there are still some

implied correlations between them.

For instance, bread and yogurt

seem exactly the opposite of each other.

It's like we are saying,

whatever is bread is not yogurt

and vice versa.

Even if this makes sense,

if we encode them in a different way.

This opposite correlation would be

true for muffins and yogurt,

but no longer for bread.

Therefore, binary encoding proves problematic,

but is a great improvement regarding

the initial ordinal method.

All right.

Finally, we have the so-called one-hot encoding.

One-hot is very simple and widely adopted.

It consists of creating as many columns

as there are possible values.

Here, we have three products.

Thus, we need three columns or three variables.

Let's call them bread, yogurt, and muffins.

Imagine these variables as asking the question:

Is this product bread?

Is this product yogurt?

And is this product muffins?

One means yes, zero means no.

So for a product that is bread,

we will have one zero zero.

For a product that is yogurt, zero one zero,

and for a product that is muffin, zero zero one.

This is very intuitive

as a product can only be of

one type at the same time.

Thus, there will be only one value one

and everything else will be zero.

This means the products are

uncorrelated and unequivocal,

which is useful and usually works like a charm.

Many lessons ago we were talking about cats,

dogs, and horses classification.

The target vectors there were one-hot encoded,

so we had the same type of vectors.

There is one big problem with

one-hot encoding, though.

One-hot encoding requires a lot of new variables.

For example, Ikea offers around 12,000 products.

Do we want to include 12,000

columns in our inputs?

Definitely not.

If we used binary, the 12,000 products

would be represented by 16 columns only

since the 12000th product would be written

like this in binary.

This is exponentially lower than the

12,000 columns we would need

for one-hot encoding.

In such cases, we must use binary,

even though that would introduce

some unjustified correlations

between the products.

Clearly there is a trade off

between binary and one-hot encoding.

We would prefer one-hot when

we have a few categories

and binary when dealing with many categories.

All right, that was all.

Thanks for watching.