All language subtitles for 13. Naive Forecast and Forecasting Metrics in Code

OK, so in this lecture, we are going to look at how to implement the Niyi forecast as well as evaluate

it using the metrics we learned about.

OK, so let's start by importing no giant pandas.

The next step is to upgrade psyche, learn the reason for this is the current version of Saika Learn

installed in Google CoLab does not have the map metric.

So as I frequently mentioned in my courses, machine learning is a field that moves fast.

This kind of thing is completely normal.

Libraries change all the time.

The next step is to import our metrics from Saikat learn so we have the map, the M80, the R-squared

and the Mzee.

You'll notice that this does not include the army nor the SMAP.

We'll see how to deal with those later.

So for this exercise, we'll be using prices from the S&P 500, which can be downloaded from my website.

The next step is to call pedigreed CSFI in order to get a data frame of our data.

As always, I'd like to do a DFAT head to get some sense for the data we're working with.

So we can see all the expected columns open, high, low and so forth.

The next step is to generate our predictions, so basically the new forecast is a forecast where we

simply predict the previous value that we can accomplish this by calling the shift function on the close

call.

And we'll call this new column close prediction.

The next step is to call DFG head once again to see our new column.

Notice that the first row now contains not a number, since, of course, there is no last value for

the first row.

OK, so for convenience, we're going to assign the true closed prices to a variable called Y true and

the predicted closed prices to a variable called a widespread.

This is what these arguments are called inside you learn.

So I felt it would be appropriate to call them the same thing.

OK, so the next portion of this notebook will be to look at our metrics, the main purpose of this

is not just to understand how to do this in code, since that's pretty easy.

But I want you to pay attention to how these values relate to each other.

Think about what would be considered a good value and what would be considered bad.

OK, so let's start with the sum of squared errors, since there's no function for this, we're going

to calculate it ourselves.

So since why shouldn't we prêt or effectively one dimensional arrays, we can just take the difference

into a dot products with the same difference.

OK, so the result is about 6000.

Of course, we don't necessarily know whether this is bad or good.

It's just a no.

The next step is to calculate the mean square error where we use our psyche to learn function.

OK, so this is the result, as you can see, this brings the number down to a more reasonable range.

Now, as Python coders, we can't be afraid of implementing things ourselves.

Some students get absolutely frightened when they see that you're not using a library.

But I urge all students not to take this approach.

In fact, implementing the MSA is trivial.

It's just what we had before, divided by the length of either flour, white, widespread.

OK, and so we get the same answer as expected.

The next step is to calculate the root mean squared error now, surprisingly, this is done by the mean

squared error function where you can pass in the argument squared equal to false.

So let's try that.

OK, so we get about one point six, seven, which makes sense.

And of course, we can just take the square root of our previous calculation, so let's try that also.

And we get the same answer as expected.

The next step is to calculate the mean absolute error, since we have a secure, learned function for

this, we are going to make use of it.

OK, so you can see that although the root mean square error and the mean absolute error have the same

units, they do not give you the same value.

Now, we know that for all the previous metrics we've seen, they are scale dependent.

So the next step is to look at the R-squared, which does not depend on the scale of the data.

OK, so this should be surprising, as you recall, we said that the best R-squared is one.

Where is the R-squared of simply predicting the mean value is zero?

It turns out that our naive forecast gets an R squared of zero point nine nine nine.

This kind of makes sense since stock prices don't vary that wildly from one day to the next.

So predicting the last value in the series should give us pretty good predictors.

However, in another sense, these are also very bad predictions because they are really just the dumbest

predictions possible.

So let this be a lesson that if you see a model that happens to predict stock prices very well, don't

assume that such a model is actually useful.

The next step is to compute the make, which is another scale and very symmetric.

OK, so this is nearly zero, which makes sense since the R-squared is nearly one.

The next step is to compute the SMAP.

Now you'll notice that I didn't import this function from Sakia learn this is because no such function

exists.

So this is the power of being able to implement these things yourself.

Someone who does not have these skills would probably start by going to Google and then checking stack

overflow and so forth.

They might end up wasting their whole day trying to figure out where is the function for SMAP.

But when you do have these skills, you're able to get this done in just a few lines of code and a few

seconds of effort.

OK, and we see that the result is pretty close to the non symmetric map, which makes sense.