CloudFormation for SysOps

AWS CloudFormation is a service that allows you to define your entire AWS infrastructure as a text file (template) in JSON or YAML format. Instead of manually clicking through the AWS Management Console or running dozens of CLI commands, you can provision a collection of resources as a single unit called a stack. This approach eliminates configuration drift, reduces human error, and enables version control of your infrastructure.

A CloudFormation template is structured into several sections. The only required section is Resources; all others are optional but commonly used.

CloudFormation provides intrinsic functions to assign values to properties at runtime. The most commonly tested ones:

CloudFormation provides pseudo parameters that are resolved automatically: - AWS::Region – the region where the stack is created. - AWS::StackId – the unique stack ID. - AWS::StackName – the stack name. - AWS::AccountId – the 12-digit account ID. - AWS::NotificationARNs – the list of SNS topics for stack events. - AWS::NoValue – used to remove a property when a condition is false.

Creating a Stack: When you create a stack, CloudFormation parses the template, validates it, and then calls the AWS APIs in the correct order based on resource dependencies (depends on attributes). For example, an EC2 instance depends on a subnet, which depends on a VPC. CloudFormation automatically determines this order from the template properties. If a resource fails to create, the default action is to roll back all successfully created resources, leaving no orphaned resources. You can disable rollback on failure using the --disable-rollback flag (CLI) or the console option.

Updating a Stack: You can update a stack by providing a new template or new parameter values. CloudFormation determines what changes are needed – it can update, replace, or delete resources. Updating can cause downtime if resources are replaced. To control the update behavior, you can use: - Change Sets: Preview the changes before applying them. - Stack Policy: Protect critical resources from accidental updates or deletion.

Deleting a Stack: Deleting a stack removes all resources that were created by the stack, in reverse order of creation. To prevent accidental deletion, you can enable termination protection on the stack. Even with termination protection, you can delete the stack by first disabling the protection.

A change set is a summary of proposed changes to a stack. It shows which resources will be added, modified, or deleted. You can create a change set, review it, and then execute it. This is useful for validating updates in a non-production environment before applying to production.

Stack sets allow you to deploy stacks across multiple accounts and regions from a single template. You define a stack set with a template and specify target accounts and regions. CloudFormation creates stacks in each target account/region. Stack sets support automatic drift detection and can be updated centrally. This is commonly used for governance (e.g., enabling AWS Config rules across all accounts in an organization).

Drift detection compares the actual state of resources in a stack with the expected state defined in the template. If someone manually modifies a resource (e.g., changes a security group rule via the console), drift detection will flag it. You can run drift detection on a stack or on individual resources. The possible drift statuses are: - DRIFTED: The actual state differs from the template. - IN_SYNC: No drift. - UNKNOWN: Drift detection has not been run or failed. - NOT_CHECKED: Drift detection has not been run on this resource.

CloudFormation uses DependsOn attribute to specify explicit dependencies. However, it can often infer dependencies from resource references (e.g., a subnet referencing a VPC ID). If you need to ensure a specific order, use DependsOn. For example, an EBS volume must be attached to an EC2 instance after the instance is created. You would set DependsOn on the volume attachment resource to the instance.

The AWS::CloudFormation::Init metadata on an EC2 instance defines configuration tasks (install packages, create files, start services). The cfn-init helper script runs on the instance and processes this metadata. This is more reliable than user data because it supports retries and can be updated without replacing the instance.

Nested stacks allow you to break a large template into smaller, reusable components. You use the AWS::CloudFormation::Stack resource type to reference another template. Nested stacks are useful for separating concerns (e.g., network layer, application layer) and for reusing common patterns (e.g., a standard VPC template).

A stack policy is a JSON document that defines which resources can be updated or deleted during a stack update. By default, all resources can be updated. You can apply a policy to protect critical resources (e.g., database instances). The policy uses Effect, Action (Update:Modify, Update:Replace, Update:Delete), and Resource (logical resource ID or wildcard).

When a stack creation or update fails, CloudFormation automatically rolls back to the last known good state. You can configure: - Rollback triggers: Monitor CloudWatch alarms and automatically roll back if an alarm goes into ALARM state during an update. - On failure: For stack creation, you can specify DO_NOTHING, ROLLBACK, or DELETE.

CreationPolicy and WaitCondition are used to pause stack creation until a signal is received. WaitCondition is a separate resource that can be used with any external signal. CreationPolicy is simpler and directly attached to a resource.

A visual tool to create and modify templates. You can drag and drop resources and see the template JSON/YAML update in real time. It helps visualize dependencies but is not required for the exam.

Custom resources allow you to implement custom provisioning logic using AWS Lambda. When CloudFormation creates, updates, or deletes a custom resource, it invokes a Lambda function that you provide. The function must return a response to CloudFormation with a status (SUCCESS or FAILED) and optional data. This is useful for integrating with third-party APIs or performing tasks not natively supported.

| Function | Purpose | Example | |----------|---------|---------| | Ref | Returns parameter value or resource physical ID | !Ref MyVPC | | Fn::GetAtt | Returns resource attribute | !GetAtt MyInstance.PublicIp | | Fn::Join | Concatenates strings | !Join ["-", ["a", "b"]] → "a-b" | | Fn::Select | Selects element from list | !Select [0, !Ref AZList] | | Fn::Sub | Substitutes variables | !Sub "${AWS::Region}" | | Fn::If | Conditional value | !If [IsProd, "t2.large", "t2.micro"] | | Fn::Equals | Equality check | !Equals [!Ref Env, "prod"] | | Fn::FindInMap | Looks up mapping | !FindInMap [RegionMap, !Ref "AWS::Region", "AMI"] |

CloudFormation requires permissions to create, describe, update, and delete resources. It uses the IAM role or user that initiates the stack operation. For cross-account stacks, you need appropriate trust policies. For stack sets, you need permissions in the target accounts. The AWS::IAM::Role can be created in the template to grant permissions to instances (e.g., for cfn-init).

When a stack update requires replacing a resource (e.g., changing an EC2 instance's security group requires replacement), CloudFormation creates the new resource first, then deletes the old one. This can cause temporary state. For stateful resources like RDS, replacement can lead to data loss unless you have snapshots.

The DeletionPolicy attribute on a resource controls what happens when the resource is deleted (either by stack deletion or update replacement). Options: - Delete (default): The resource is deleted. - Retain: The resource is retained (e.g., to keep an S3 bucket with data). - Snapshot: For supported resources (e.g., RDS, EC2 volume, ElastiCache), a snapshot is taken before deletion.

Similar to DeletionPolicy but applies when a resource is replaced during an update. Options: Delete, Retain, Snapshot.

Common attributes you can set on any resource: - Condition: Reference a condition from the Conditions section. - DependsOn: Explicit dependency. - CreationPolicy: Wait for signals. - UpdatePolicy: Update behavior for Auto Scaling groups. - DeletionPolicy: What to do on delete. - UpdateReplacePolicy: What to do on replace. - Metadata: Arbitrary data.

You can validate a template using the aws cloudformation validate-template command. It checks syntax and resource types but does not verify that the resources will be created successfully (e.g., it won't check for insufficient IP addresses).

Every stack operation generates events. You can view them in the console or via CLI (aws cloudformation describe-stack-events). Events include resource creation, update, and deletion statuses, along with timestamps and logical IDs.

When you use a CreationPolicy on an Auto Scaling group or EC2 instance, you must run cfn-signal on the instance after configuration is complete. The command sends a success or failure signal. Example:

cfn-signal --stack MyStack --resource MyAutoScalingGroup --region us-east-1

If the signal is not received within the timeout, stack creation fails.

Common commands: - aws cloudformation create-stack --stack-name my-stack --template-body file://template.yaml --parameters ParameterKey=KeyName,ParameterValue=mykey - aws cloudformation update-stack --stack-name my-stack --template-body file://updated.yaml - aws cloudformation describe-stacks --stack-name my-stack - aws cloudformation delete-stack --stack-name my-stack - aws cloudformation create-change-set --stack-name my-stack --template-body file://new.yaml --change-set-name my-change-set - aws cloudformation execute-change-set --change-set-name my-change-set --stack-name my-stack - aws cloudformation detect-stack-drift --stack-name my-stack

Drift detection results include: - StackDriftStatus: DRIFTED, IN_SYNC, UNKNOWN, NOT_CHECKED. - DriftedStackResourceCount: Number of drifted resources. - Timestamp: When detection was completed.

While Terraform is not on the SOA-C02 exam, understanding the differences can help. CloudFormation is AWS-native and deeply integrated with IAM and other services. It uses declarative JSON/YAML. Terraform uses HCL and supports multiple providers. For the exam, focus on CloudFormation-specific features like change sets, stack sets, drift detection, and helper scripts.

A large enterprise uses AWS Organizations to manage 50 accounts across multiple business units. The security team wants to enforce a baseline of security groups, AWS Config rules, and CloudTrail trails in every account. They create a CloudFormation stack set with a template that defines these resources. The stack set is configured with service-managed permissions, automatically deploying to all accounts in the organization. When a new account is added, CloudFormation automatically deploys the stack to it. If a security group is manually modified in one account, drift detection flags it. The security team runs drift detection weekly and remediates any drifted stacks by updating the stack set, which re-deploys the correct configuration. This ensures consistent security posture across all accounts without manual intervention.

A SaaS company runs hundreds of microservices on ECS. Each service has its own CloudFormation template that defines ECS tasks, services, load balancers, and auto scaling. The team uses AWS CodePipeline to automate deployments. When a developer pushes code to a Git repository, CodePipeline triggers a build, runs tests, and then uses CloudFormation to update the stack for that service. The template uses parameters for the Docker image tag, so each deployment updates the task definition. Change sets are used to preview the update before applying. If the update fails (e.g., due to a misconfigured health check), CloudFormation rolls back, and the pipeline fails, alerting the team. This approach allows safe, automated deployments with minimal downtime.

A financial institution needs to replicate its production environment in a secondary region for disaster recovery. They create a master template that calls nested stacks for each layer: networking, compute, database, and application. The nested stacks are parameterized by environment (prod, dr). The DR stack is created in the secondary region with different parameter values (e.g., smaller instance types). The master template uses Fn::ImportValue to share outputs between nested stacks (e.g., VPC ID from the networking stack is imported by the compute stack). When a DR test is required, the team updates the DR stack with the latest production parameters, runs drift detection to ensure consistency, and then tests failover. This modular approach allows independent updates to each layer and reusability across regions.