Original subtitles

OK, so in this lecture, we are going to investigate the Box Cox and other Times series transformations

in code.

So we'll start by downloading a famous time series data set called Airline Passengers.

Next, we're going to import numbers, pandas and matplotlib, nothing you haven't seen before, and

also the Buzzcocks function from Zippi.

Next, we're going to read an RSV using Pedigree's CSFI.

So, again, nothing too surprising here.

Now, one thing I always like to do whenever I load in some data is to just see what it looks like.

This always makes me more confident in the code that I wrote, and it lets me know that my code makes

sense.

So we'll do a DFT head and this will print out the first few rows of the data.

OK, so we can see that for the index, we have the date, which is monthly, and we have one integer

column called the passengers.

The next step is to plot our data set just to see what we're dealing with.

So there are several characteristics of this data that I want you to notice, no one noticed that it

has a trend.

This Time series is going upward into the right number to notice that it has some seasonality.

That is, there is a repeating pattern in time.

Number three, notice that the amplitude of this seasonal pattern increases over time.

So at the beginning, the amplitude is pretty small, but at the end it gets larger and larger.

So these are all characteristics of Time series that we will explicitly model in this course.

And you'll learn how each algorithm handles these characteristics.

One thing we would like to see, which will make more sense later, is for things to not change over

time.

So, for example, this amplitude increasing over time.

It would be nice if that went away.

OK, so the next step will be to try to square root transform.

So here we call the security function on the passengers column.

The next step is to plot our new column.

OK, so we can see that the Time series has been squashed down slightly, but the amplitude of the seasonal

pattern still seems to increase over time.

The next step is to try the log transform.

So here we call the log function on the passengers column.

The next step is to plot our new column.

OK, so this log transform seems to do a pretty good job at squashing down the data to make it look

more uniform and time.

The final step will be to do a box cox transform, so this function takes in a one dimensional data

set as input and it returns the transform data along with the optimal value of lambda.

So you can see we've assigned the result to the variables called data in Lambe.

The next step is to just print out lamb to see what value we got.

So we get about zero point one five, which is kind of in between the log transform in the square root

transform, the next step is to assign our box --'s transform data to a new column in our data frame

and make a new plot.

So let's try that.

OK, and we see that it definitely looks like something in between the square root and the log transforms.

The next step is to visualize our data in the form of a histogram.

This should give us some insight into what the box clocks transform actually does.

But as mentioned in the theory lecture, keep in mind that this kind of plot doesn't really make sense

in terms of the distribution of the data.

We can't really talk about the distribution when the distribution is dynamic and changing in time.

OK, so for the raw passenger's data, we see that most of the values are concentrated in the lower

hundreds.

Now for the square root data, we see that the distribution has been pushed further to the right, so

it's more flat than before and less concentrated on the lower values.

For the log data, we see that the distribution now kind of resembles a mountain where it's more evenly

spaced out in the center instead of off to one side.

And for the Box Cox data, we see almost the same pattern, except the largest peak is now closer to

the center.

OK, so that's pretty much it for this code.

I think this is enough to give you some sense of what effect these different transformations have.

Note that in this course, we won't be applying the box cox transform very often the reason for this

is there's going to be a combinatorial explosion of techniques for us to try.

And if we tried them all, not only would you get very bored and think this course was very tedious,

you wouldn't gain very much.

But I want to make you aware of these tools so that you can apply them in your work if you think they

would be useful.