All language subtitles for 05_regularized-logistic-regression.en

In this video, you see how to

implement regularized
logistic regression.

Just as the gradient update
for logistic regression has

seemed surprisingly similar to

the gradient update
for linear regression,

you find that the
gradient descent update

for regularized logistic
regression will

also look similar to

the update for regularized
linear regression.

Let's take a look.
Here is the idea.

We saw earlier that

logistic regression can
be prone to overfitting

if you fit it with
very high order

polynomial features like this.

Here, z is a high order
polynomial that gets passed into

the sigmoid function like so
to compute f. In particular,

you can end up with a
decision boundary that is

overly complex and
overfits as training set.

More generally, when you train

logistic regression
with a lot of features,

whether polynomial features
or some other features,

there could be a higher
risk of overfitting.

This was the cost function
for logistic regression.

If you want to modify it
to use regularization,

all you need to do is add
to it the following term.

Let's add lambda to
regularization parameter over

2m times the sum from
j equals 1 through n,

where n is the number of
features as usual of wj squared.

When you minimize
this cost function

as a function of w and b,

it has the effect of
penalizing parameters w_1,

w_2 through w_n,

and preventing them
from being too large.

If you do this, then even
though you're fitting

a high order polynomial
with a lot of parameters,

you still get a decision
boundary that looks like this.

Something that looks
more reasonable

for separating positive
and negative examples

while also
generalizing hopefully

to new examples not
in the training set.

When using regularization,

even when you have
a lot of features.

How can you actually
implement this?

How can you actually minimize
this cost function j of

wb that includes the
regularization term?

Well, let's use gradient
descent as before.

Here's a cost function
that you want to minimize.

To implement gradient
descent, as before,

we'll carry out the following
simultaneous updates

over wj and b.

These are the usual update
rules for gradient descent.

Just like regularized
linear regression,

when you compute where there
are these derivative terms,

the only thing that
changes now is that

the derivative respect to wj
gets this additional term,

lambda over m times wj
added here at the end.

Again, it looks a lot like

the update for regularized
linear regression.

In fact is the exact
same equation,

except for the fact
that the definition of

f is now no longer
the linear function,

it is the logistic
function applied to z.

Similar to linear regression,

we will regularize only
the parameters w, j,

but not the parameter b,

which is why there's no change

the update you will make for b.

In the final optional lab of

this week, you
revisit overfitting.

In the interactive plot
in the optional lab,

you can now choose to
regularize your models,

both regression and
classification,

by enabling
regularization during

gradient descent by selecting
a value for lambda.

Please take a look
at the code for

implementing regularized

logistic regression
in particular,

because you'll implement this in

practice lab yourself at
the end of this week.

Now you know how to implement

regularized logistic regression.

When I walk around
Silicon Valley,

there are many
engineers using machine

learning to create
a ton of value,

sometimes making a lot of
money for the companies.

I know you've only been studying

this stuff for a few weeks but

if you understand and can

apply linear regression
and logistic regression,

that's actually all
you need to create

some very valuable applications.

While the specific
learning outcomes

you use are important,

knowing things like
when and how to reduce

overfitting turns
out to be one of

the very valuable skills
in the real world as well.

I want to say congratulations

on how far you've
come and I want

to say great job
for getting through

all the way to the
end of this video.

I hope you also work through

the practice labs and quizzes.

Having said that,
there are still

many more exciting
things to learn.

In the second course of
this specialization,

you'll learn about
neural networks,

also called deep
learning algorithms.

Neural networks are
responsible for

many of the latest
breakthroughs in the eye today,

from practical speech
recognition to computers

accurately recognizing
objects and

images, to self-driving cars.

The way neural
network gets built

actually uses a lot of what
you've already learned,

like cost functions,

and gradient descent,
and sigmoid functions.

Again, congratulations
on reaching

the end of this third and
final week of Course 1.

I hope you have [inaudible]
and I will see you

in next week's material
on neural networks.