Monitoring with Amazon CloudWatch

Amazon CloudWatch is a monitoring service for AWS resources and applications. Think of it as a dashboard that collects data about everything happening in your cloud: how much CPU your EC2 instance uses, how many requests your load balancer handles, and even custom business data like the number of shopping cart checkouts per minute.

The first core concept is a metric. A metric is a specific data point measured over time — for example, the CPU utilisation of a virtual server. Metrics are organised into namespaces, which are like folders that keep data from different services separate, like the EC2 namespace or the Lambda namespace. Each metric has a timestamp, a value, and optionally dimensions. Dimensions are key-value pairs that describe the metric, like InstanceId=i-1234567890abcdef0. They let you filter and drill down. For the exam, you need to know that CloudWatch stores metrics for 15 months, but the default retention is 15 months (you cannot change it, but you can get more frequent data with high-resolution metrics).

Next is the alarm. An alarm watches a single metric over a specified time period. You define a threshold (like CPU utilisation greater than 80%) and a period (like evaluate every 5 minutes). When the metric crosses the threshold for a certain number of consecutive evaluation periods (called datapoints to breach), the alarm changes state from OK to ALARM. Alarms can trigger actions: send a notification via Amazon Simple Notification Service (SNS), or automatically stop, terminate, reboot, or recover an EC2 instance. The exam loves to test the three states: OK, ALARM, and INSUFFICIENT_DATA — the last means there is not enough data to decide.

Then there is logs. CloudWatch Logs collects, monitors, and stores log files from EC2 instances, Lambda functions, and other sources. A log event is a single record of activity (like an error message or an access request). Log events are grouped into log streams, which are sequences of events from the same source (like one EC2 instance). Log streams are collected into log groups, which share the same retention, monitoring, and access control settings. You can set up metric filters to turn log data into CloudWatch metrics — for example, count every occurrence of the word ERROR in your application log and create a new custom metric. This is powerful for troubleshooting.

CloudWatch also includes dashboards. A dashboard is a customisable view in the CloudWatch console that displays metrics and alarms in widgets. You can create multiple dashboards for different teams or environments. They are great for operational visibility.

Finally, there is CloudWatch Events (now called Amazon EventBridge, but the exam may still use the old name). It allows you to respond to state changes in your AWS resources. For example, if an EC2 instance stops, you can trigger a Lambda function to start it back up. It is separate from alarms but often conflated.

Why does this exist? Before cloud, monitoring meant installing software on each server, setting up a central server to collect data, and praying the network did not fail. CloudWatch removes that management overhead. You do not install anything — it is built into AWS. It replaces expensive, self-managed monitoring stacks and gives you a single pane of glass for all your resources.

For the SOA-C02 exam, you must understand how to set up basic monitoring (enabled by default for some services), detailed monitoring (provides 1-minute metrics at extra cost), create alarms, and configure log collection via the CloudWatch agent or via AWS services directly. You also need to know how to use metric math to combine multiple metrics into a single alarm.

An IT professional at a startup called 'QuickCart' — an online grocery delivery service — uses CloudWatch every day to keep the site running. Here is what that looks like.

QuickCart runs its application on a fleet of Amazon EC2 instances behind an Application Load Balancer. The ops team's number one worry is the site going down during peak hours, like Saturday at 11 a.m. when everyone orders lunch.

The first step is to create a dashboard. The senior sysadmin logs into the AWS Management Console, navigates to CloudWatch, and creates a new dashboard called 'QuickCart Production'. They add widgets: a line graph of CPU utilisation for all EC2 instances, a number widget showing the current request count per minute on the load balancer, and a text widget with the on-call phone number. This dashboard is pinned to a monitor in the office.

The second step is setting up alarms. The sysadmin creates an alarm on the metric 'CPUUtilization' for each instance. The threshold is 80% for 2 consecutive 5-minute periods. The action is to send a message to an SNS topic that triggers an email to the team and also pages the on-call engineer. They also create an alarm on the load balancer's 'HTTPCode_ELB_5XX_Count' metric — if error rates spike, the team knows immediately.

The third step involves logs. QuickCart's application runs on a custom web server that writes logs to disk. The sysadmin installs the CloudWatch agent on each EC2 instance. The agent is a piece of software that reads local log files and sends them to CloudWatch Logs. They configure it to monitor /var/log/app/error.log and send those log events to a log group called '/quickcart/app-errors'. They then create a metric filter on that log group that counts every line containing 'FATAL'. That count becomes a custom metric, and they create an alarm on it.

The fourth step is reaction. One Tuesday, the alarm fires. CPU utilisation on one instance hits 95%. The on-call engineer receives the SNS notification on their phone. They check the dashboard and see a spike. They then look at the log group and search for recent events using CloudWatch Logs Insights (a query tool). They discover a memory leak in a new deployment. They roll back the application and the alarm clears.

What tools does the professional use? They use the AWS Management Console for setup, the AWS CLI for scripting alarm creation, the CloudWatch agent for log collection, and CloudWatch Logs Insights for ad-hoc querying. - They set up automatic actions: one alarm triggers an Auto Scaling policy to add more instances. - They create a custom dashboard per environment (dev, test, prod). - They set up a billing alarm to alert if the monthly spend goes over budget. - They use composite alarms (which combine multiple other alarms) to reduce noise — only alert if CPU is high AND error rate is high.

Without CloudWatch, QuickCart would not know about problems until customers complain, which is too late. CloudWatch gives them proactive awareness. For the exam, you must be able to describe this exact workflow: install agent, send logs, create metric filter, alarm, action.