SSM Automation Runbooks

AWS Systems Manager Automation is a service that enables you to safely automate common and repetitive IT operations and management tasks. An Automation Runbook is a document that defines the actions that Systems Manager performs on your AWS resources. Runbooks can be either predefined by AWS (owned by Amazon) or custom-written by you. They are written in YAML or JSON and consist of a series of steps, each of which can invoke an AWS API action (like aws:executeAwsApi), run a script on an instance (aws:runCommand), or perform conditional logic (aws:branch).

Before Automation, SysOps engineers had to manually perform repetitive tasks like patching instances, creating AMIs, or responding to CloudWatch alarms. This was error-prone, time-consuming, and difficult to audit. Automation Runbooks solve this by providing a declarative, version-controlled, and auditable way to define and execute operational procedures. They integrate with IAM for fine-grained permissions, CloudTrail for logging, and CloudWatch Events (now Amazon EventBridge) for triggering based on events.

When you start an Automation execution, the Systems Manager service reads the runbook document and begins executing steps sequentially. Each step has an action that maps to an AWS API call or a Systems Manager command. The service uses the IAM role associated with the execution (the Automation Assume Role) to perform actions on your behalf. For steps that target EC2 instances, the SSM Agent on the instance executes the command and returns the output. The Automation service tracks the status of each step (Success, Failed, TimedOut, Cancelled) and can branch based on conditions. If a step fails, the runbook can be configured to stop, continue, or roll back to a previous step via the onFailure property.

To create a custom runbook, you can write a document and store it in Systems Manager Documents (under Automation). Example CLI command to create a document:

aws ssm create-document \
  --content file://myRunbook.yaml \
  --name "MyCustomRunbook" \
  --document-type "Automation"

To start an execution:

aws ssm start-automation-execution \
  --document-name "MyCustomRunbook" \
  --parameters "key=value" \
  --target-parameter-name "InstanceId" \
  --targets "Key=tag:Environment,Values=Production"

To check execution status:

aws ssm describe-automation-executions --filter "Key=DocumentName,Values=MyCustomRunbook"
aws ssm get-automation-execution --automation-execution-id "execution-id"

Each step in a runbook can be one of many action types. Common ones include:

Steps can also use outputs to capture data from one step and pass it to subsequent steps via {{ stepName.outputName }}.

Automation executions are logged in CloudTrail. You can use IAM policies to restrict who can start executions and which documents they can use. The Automation Assume Role must have permissions for all actions in the runbook. Use aws:SourceAccount and aws:SourceArn in the role trust policy to prevent confused deputy problems.

A large enterprise runs hundreds of EC2 instances across multiple accounts. They need to create weekly AMIs of all instances for disaster recovery, while also deregistering AMIs older than 90 days. Using a custom Automation Runbook, they automate the entire process. The runbook first stops the instance (if required), creates an AMI with tags (e.g., CreationDate, Owner), then restarts the instance. It also deletes old AMIs and associated snapshots. The runbook is triggered weekly by an EventBridge schedule. Rate control is set to maxConcurrency=10 and maxErrors=5 to avoid overwhelming the account. The runbook uses aws:executeAwsApi to call ec2:CreateImage and ec2:DeregisterImage. IAM policies are scoped to specific resources. Common misconfiguration: forgetting to exclude instances with a 'NoBackup' tag, causing unnecessary downtime. The solution uses a branching step to check tags before proceeding.

A financial services company must patch all Windows and Linux instances monthly. They use the AWS-provided runbook AWS-RunPatchBaseline with custom parameters. The runbook scans for missing patches, installs them, and reboots if required. To avoid business impact, they use an approval step before the reboot. The runbook targets instances by the tag 'PatchGroup: Monthly'. The Automation Assume Role is scoped to allow ec2:RebootInstances only on tagged instances. They also integrate with AWS Config: if an instance falls out of compliance (e.g., missing critical patches), a Config rule triggers a remediation runbook that patches the instance immediately. A common pitfall is not setting the reboot parameter correctly, causing instances to reboot unexpectedly or not reboot when needed. The team uses maxAttempts: 3 on the patch step to handle transient errors.

A SaaS company uses CloudWatch alarms for high CPU utilization. When an alarm triggers, an EventBridge rule starts an Automation Runbook that captures a memory dump, stops the instance, attaches a diagnostic volume, runs analysis scripts, and then restarts the instance. The runbook includes an approval step before stopping the instance to allow a human to intervene if it's a critical production server. The runbook outputs the analysis results to an S3 bucket. The team learned the hard way that the Automation Assume Role must include permissions for ec2:StopInstances and ec2:CreateVolume, and that the SSM Agent must be installed on the instance. They also set timeoutSeconds on the analysis step to 600 to avoid hanging.