All language subtitles for 22 - CloudWatch Unified CloudWatch Agent Hands On English

Okay, so in this lecture

we're going to install the CloudWatch unified agent

to collect metrics and log for Amazon EC2 instances.

So to do so, very simple

we're first going to have to create an IAM role.

And this role will allow us to send metrics

and logs to CloudWatch.

So to do so let's go into roles, let's create a role

and this is going to be a role for AWS service

which is an EC2 instance.

So we need to create an EC2 instance role

and then we click on next.

So EC2, just select EC2 here.

Next, we need to find permission policies

to apply to this role.

So for this I'm going to type CloudWatchAgent

and we have two

we have the Server Policy and the Admin Policy.

The one we're going to use right now

is the CloudWatchAgentServerPolicy.

And the reason we do so is that

we want to be able to put metric data into CloudWatch

also to send logs

and finally to get parameters from SSM

because it is possible for us to retrieve a CloudWatch

unified agent configuration directly from SSM

as we will see in this lecture.

So this is enough. Let's click on next.

And then the role name should be, for example

Amazon EC2 role for CloudWatch.

Okay. Let's create this role.

And we're done.

So next what I have to do is

to create an EC2 instance

that will be leveraging this role.

So to do so let's go into the management console

into the EC2 console.

And then I'm going to create an EC2 instance.

So I'll go into launch instance and we'll configure it.

So we're going to use Amazon Linux 2 AMI

and then I will use t2.micro.

We don't need to select a key pair

if we don't want to, which is fine.

We'll use EC2 instance connect

to connect to our EC2 instance.

Then we need to create a new security group

and we'll allow SSH and HTTP traffic

the third one, from the internet

because we'll be launching a web server

and we'll get the log from that server.

So we allow SSH to SSH to our instance

and HTTP to allow port 80 traffic on to our EC2 instance.

This is good.

And then we're good to go.

Let's launch this instance.

Okay. So let's click on it

and we're gonna find the instance right now.

So let me wait for it to be ready.

Okay. So our EC2 instance is now created.

Let's connect to it

using the EC2 instance user connect interface.

So we establish a connection directly into the instance

and now I'm able to issue a few commands.

So the first one I'm going to do

is to elevate my privileges.

So I will do sudo su and now I am a super user

on my instance.

And I will do yum install HTTPD.

This will install a web server and that web server

I say yes.

That web server is going to have traffic

coming from the internet.

And we'll take the log out of HTTP

to connect to our instance

to send that to CloudWatch logs

So the HTTPD service is installed.

And now we're just going to echo "hello world"

into var/www/html/index.html.

So now we have an index HTML file created.

And we need to just start our HTTPD server.

So for this, we'll do sudo systemctl start httpd

This will effectively start our HTTPD service.

And if wanted to allow also it to survive through restarts

we will enable it.

So now back into here, this is the public IP

of my EC2 instance, which is great.

And so if I visit it

now I get the prompt hello world.

So that means that we are indeed able

to access our EC2 instance using

this Apache web server.

And now what I want to show you is that

out of this, this created some log entries.

So if we go into the file

so I'll do cat /var/log/httpd

and then we have access underscore log.

This contains the request log of all

the request made into our server.

As you can see, we can find the fact that

yes we did do a get right now.

So also, should there be any errors

we can do /error_log

and we'll get a list of all the errors

that are happening for our HTTPD server.

So these two logs, the error log

as well as the access log

are the logs that I want to send into CloudWatch logs

using the CloudWatch unify agents.

So to do so, we can follow documentation

but this is quite complicated.

So it's to collect metrics and logs

for Amazon EC2 instances

and I've summarized all the instructions in the code.

So the first thing we have to do is to install the agents

on Amazon Linux 2

and for this we can just run this command

a sudo yum install of the Amazon CloudWatch agent.

So, I say, yes and it's going to be installed.

The next thing we have to do is to run a wizard.

And this is to configure the CloudWatch agent.

This is something we have to do once

and then we'll store the configuration

in the SSM parameter restore.

And it will be retrieved at run time if we need to.

So let's go ahead and clear this

and we're going to run the wizard.

So now it says

on which OS are you planning to use the agent wave?

So Linux.

Then is this EC2 or On-Premises?

This is EC2.

We're going to run this as the route user.

And do you want to turn on StatsD daemon?

Why not? Yes.

So I press one every time.

What port do you want this daemon to listen to?

So we'll just keep the default choice.

Then what is the correct interval?

We'll just keep it as default as well.

So that means that the CloudWatch unified agent

can also collect stuff from StatsD.

Then we have, do we want to have aggregations?

Yes. We'll aggregate them at 60 seconds interval

which is the default.

Do you want to collect metrics from CollectD?

And it must be installed or the agent will fail to start.

So it turns out that I don't have CollectD

on this instance, but it is possible for us

to enable this if we wanted to

but because if we don't have it installed

the agent will fail to start.

We're just going to reply no to this answer.

So I will enter two.

Do you want to monitor any host metrics

such as CPU, memory etcetera?

Yes.

And as you can see, using the CloudWatch unified agent

I am able to have access to memories.

So Ram is a metric you can access

from within the EC2 instance.

We can also collect CPU metrics at the core level.

Again, something this is only possible

by doing metric collection from within the EC2 instance.

Do you want to add EC2 dimensions?

Absolutely. Yes.

And do you want to aggregate them?

Why not?

Then we have the high resolution metrics.

So in case we have that

we have the ability to customize

which resolution will go on them.

So one second, ten seconds, 30 seconds or 60 seconds.

Now to keep cost under control

I will keep it as 60 seconds.

As you can see we can have a resolution

down to one second.

Now, which default metrics config do you want?

Do you want basic, standard, advanced or none?

So we'll keep the default as basic.

And here now I have an entire JSON document

of the configuration we just did.

Which summarizes all the answers we have selected

for the configuration of this CloudWatch unified agent.

So this is good enough. You can have a look.

Are we satisfied with the config?

The answer is yes.

Now, do we have any existing CloudWatch log agents

from configuration file for migration?

The answer is no.

So there used to be before the unified agent

there used to be an old thing called

the Amazon CloudWatch log agents.

And if you had that we could import it

but we don't have this right now.

So we'll say no.

And as you can see

now the unified agents does both metrics

and log at the same time.

Now, do you want to monitor any log files?

We'll say yes.

And now we need to enter the log file path.

So for this is going to be var/log/httpd/access_log

which is the first log file I showed you.

And the group name is going to be access_log for it.

And the log screen name is going to be perfect as well.

And the log retention in days

we could do minus one, for infinity

or we could say, for example, five for seven day retention.

And do you want to additionally specify other log files?

The answer is yes. I want to.

So we enter the path again.

So var/log/httpd/error_log.

Default group name we will keep.

Default stream we keep.

And then again, we'll have a seven day retention

so I'll just enter number five.

Okay. Do you want to specify any more logs we've done?

So just no.

And now we have another config right here.

So now this contains some configuration

for the logs, for our instance.

So this is good enough and I'm good to go.

So do you want to store this config now

in the SSM parameter store?

We will answer yes.

But it will not work

until we have the correct permissions.

So currently this config file has been located and stored

at this address right here, which is great for us

but we want to send this into the SSM parameter store

so that it can be retrieved

by other EC2 instances at run time.

So to do so, we need to store the config.

We need to store the config, this one

in the SSM parameter store

And to do so we need the right permissions.

But if we go into the role we created

the Amazon EC2 role for CloudWatch

which by the way we did not attach to the instance.

Now that I just remembered it.

So let's go back here.

And of course, we'll do security

and then modify IAM role

and we'll need to attach

this Amazon EC2 role for CloudWatch that I just created.

So you see, comes full circle.

So this is perfect.

Even this role,

if we look at the permission itself

and we look at SSM

it's allowed to get a parameter

but not to send a parameter.

So on one instance, we need to add the permissions

to attach a policy

and I will just type CloudWatch agent again

and this time we'll attach the admin policy

which has sufficient permission to put the parameter

into SSM.

And this is only needed when you do the setup

of the CloudWatch unified agents.

So we'll attach this policy. Now we have two policies.

And now my EC2 instance

should be able to perform this next action.

So I will say, yes.

Do you want to store it into the SSM parameter store?

So, yes.

And then what parameter name do you want to use?

So we can just use the default choice.

Amazon Linux, Amazon CloudWatch, minus Linux, press enter.

And then which region do you want to store it in?

So EUS one, which was retrieved directly from

the EC2 metadata by this wizard.

Perfect.

Now which credential should be used?

So the one from the SDK. I press enter.

And then it says successfully put config to parameter store

as Amazon CloudWatch Linux and the program exits now.

So that means that if we go into Amazon Web Services

and then we go into the SSM parameter store.

So let's go into systems manager.

And then find the parameter store.

As we can see now I have one standard parameter

called the Amazon CloudWatch Linux.

And if I click on it

and I look at the value, as you can see

the value is this entire JSON file

that was generated by this wizard.

So it's amazing.

So now the wizard is running and then this has been inserted

into the SSM parameter store.

And now we want to boot up from this parameter.

So back onto our instance, let's clear this screen.

So let's assume this is a whole new instance, okay?

And what we want to do is to just boot up

from this configuration from the SSM parameter store

and start sending metrics and logs into CloudWatch.

So to do so let's go back into our code.

And for this, we have a few options.

So this one can be deleted

because we don't use the CollectD.

So we have options here.

So we can either start directly

from the SSM parameter store.

And this is this command right here.

So the Amazon CloudWatch agent CTL to fetch a config

from SSM at this address right here.

And it's going to fetch the config

we have inserted right here.

So this should work.

The other option is to instead provide a flat file.

So config the JSON file that has been generated

using the wizard, for example

and then the agent can start the quiz

from this static configuration file.

But if you plan to use multiple instances

start from the same file then

it may be a good idea to store it into SSM.

So I will use the command that boots up from SSM.

I will paste it, press enter.

And as you can see, it says

that the configuration version is succeeded

and then it should be good to go.

So let's have a look and see if anything works.

So let's go into the CloudWatch agents.

So let's go into the CloudWatch service.

So CloudWatch.

Perfect.

Let's make sure to use the same region

my EC2 instance is in, which is EU-west-1.

And then I will go into log groups

and underneath we find access log and error log

which are the two logs created by my unified agents.

So under access log, I can find one log stream

which represents my EC2 instance.

And as you can see

we have the two events we saw from before.

Now the cool thing is that if I refresh this page right here

one or two or three times

at some point

the events are going to end up in this log.

So as you can see now

the events have been written by the CloudWatch log agents

unified agents

into this log right here, which is amazing.

And again, if you went into the error log

you will find the log stream right here

that contains the error log stream, if we need to.

So this is perfect.

Now what about metrics?

So if we go into all metrics, we can find a name space

custom name space called CloudWatch agents, CW agents.

And then we can go, for example, by instance ID

and here find the memory used percent

which represents how much ram is being used

by my EC2 instance.

Or the disc used percent and so on.

And you can explore the different metrics.

Here we have the mem used percent again,

but by image ID and with other dimensions and so on.

And here we have six metrics available to us

that are right here.

And it's possible for you to explore the documentation.

And here you can find at some point

you can find all the metrics

that are collected by the CloudWatch agents

they're right here

and the explanations are self explicit.

But you can, just to summarize

collect some CPU metrics, which are very handy.

You can collect as well, some disc space

and IO and so on for your disc.

You can also get information around your mem

so active, available, cached, free and so on.

This is very helpful.

Some network information.

So a number of packets in and out and so on.

And then information about your processes

running on your EC2 instances.

So how many are idle, how many you're running

et cetera, et cetera, which is very, very helpful.

And finally, the swap that you have

on your machine free, used and so on.

So that's it, we've successfully configured

the CloudWatch AI unified agent.

We stored the configuration in SSM and retrieved it

and we're good to go.

Okay. So just to be done, if you are in the DevOps course

you need to keep this instance running.

But if you are in the CSOPs course

then you can terminate this instance

and you'll be good to go.

All right, that's it.

I hope you liked it.

And I will see you in the next lecture.