All language subtitles for 01_gradient-descent-implementation.en

To fit the parameters of a
logistic regression model,

we're going to try to find

the values of the
parameters w and b

that minimize the cost
function J of w and b,

and we'll again apply
gradient descent to do this.

Let's take a look at how.

In this video we'll
focus on how to find

a good choice of the
parameters w and b.

After you've done so,

if you give the model
a new input, x,

say a new patients at

the hospital with a certain
tumor size and age,

then these are diagnosis.

The model can then
make a prediction,

or it can try to estimate

the probability that
the label y is one.

The average you can
use to minimize

the cost function is
gradient descent.

Here again is the cost function.

If you want to minimize

the cost j as a
function of w and b,

well, here's the usual
gradient descent algorithm,

where you repeatedly update

each parameter as the
0 value minus Alpha,

the learning rate times
this derivative term.

Let's take a look
at the derivative

of j with respect to w_j.

This term up on top
here, where as usual,

j goes from one through n,

where n is the
number of features.

If someone were to apply
the rules of calculus,

you can show that the derivative
with respect to w_j of

the cost function capital J is

equal to this
expression over here,

is 1 over m times the sum

from 1 through m of
this error term.

That is f minus the
label y times x_j.

Here are just x I j is

the j feature of
training example i.

Now let's also look
at the derivative of

j with respect to
the parameter b.

It turns out to be this
expression over here.

It's quite similar to
the expression above,

except that it is
not multiplied by

this x superscript i
subscript j at the end.

Just as a reminder,

similar to what you saw
for linear regression,

the way to carry out
these updates is

to use simultaneous updates,

meaning that you first

compute the right-hand
side for all of

these updates and
then simultaneously

overwrite all the values on
the left at the same time.

Let me take these
derivative expressions

here and plug them
into these terms here.

This gives you gradient descent
for logistic regression.

Now, one funny
thing you might be

wondering is, that's weird.

These two equations look

exactly like the
average we had come up

with previously for
linear regression

so you might be wondering,

is linear regression actually

secretly the same as
logistic regression?

Well, even though these
equations look the same,

the reason that this is not
linear regression is because

the definition for the
function f of x has changed.

In linear regression,

f of x is,

this is wx plus b.

But in logistic regression,

f of x is defined to be

the sigmoid function
applied to wx plus b.

Although the algorithm
written looked the

same for both linear regression
and logistic regression,

actually they're two very
different algorithms

because the definition for
f of x is not the same.

When we talked about
gradient descent

for linear regression
previously,

you saw how you can monitor

a gradient descent to
make sure it converges.

You can just apply
the same method for

logistic regression to make
sure it also converges.

I've written out these
updates as if you're updating

the parameters w_j one
parameter at a time.

Similar to the discussion

on vectorized implementations
of linear regression,

you can also use
vectorization to make

gradient descent run faster
for logistic regression.

I won't dive into the details of

the vectorized implementation
in this video.

But you can also learn
more about it and

see the code in
the optional labs.

Now you know how to implement

gradient descent for
logistic regression.

You might also remember feature

scaling when we were
using linear regression.

Where you saw how
feature scaling,

that is scaling all
the features to

take on similar
ranges of values,

say between negative
1 and plus 1,

how they can help gradient
descent to converge faster.

Feature scaling
applied the same way

to scale the different
features to take on

similar ranges of
values can also speed

up gradient descent for
logistic regression.

In the upcoming optional lab,

you also see how the gradient

for the logistic regression
can be calculated in code.

This will be useful to
look at because you

also implement this in

the practice lab at
the end of this week.

After you run gradient
descent in this lab,

there'll be a nice set of

animated plots that show
gradient descent in action.

You see the sigmoid function,

the contour plot of the cost,

the 3D surface plot of the cost,

and the learning curve or

evolve as gradient descent runs.

There will be another
optional lab after that,

which is short and sweet,

but also very useful

because they're
showing you how to use

the popular
scikit-learn library to

train the logistic regression
model for classification.

Many machine learning
practitioners in

many companies today use

scikit-learn regularly
as part of their job.

I hope you check out
the scikit-learn

function as well and

take a look at how that
is used. That's it.

You should now know how to
implement logistic regression.

This is a very powerful and very

widely used learning algorithm

and you now know how to get it

to work yourself.
Congratulations.