# CloudFormation Custom Resources with Lambda

> Chapter 99 of the Courseiva DVA-C02 curriculum — https://courseiva.com/learn/dva-c02/cfn-custom-resources

**Official objective:** 3.2 — Deployment

## Introduction

AWS CloudFormation Custom Resources serve as a powerful mechanism to extend CloudFormation's provisioning capabilities beyond native resources. For the DVA-C02 exam, Custom Resources are a high-frequency topic within the Deployment domain (Objective 3.2: Deploy and manage resources with AWS CloudFormation). Expect 1-2 questions specifically on Custom Resource lifecycle, Lambda integration, and failure handling. Mastering this topic requires understanding the request/response protocol, timeout constraints, and common pitfalls like missing signal responses.

## Custom Resources: The CloudFormation Swiss Army Knife

A construction site, a crew of builders with standard tools. The crew has a standard set of tools for framing, wiring, and plumbing — these are the built-in AWS resources. But what if you need a custom-made chandelier that requires a specialized artisan? You can't build it with standard tools. So, you hire a freelance artisan (Lambda function) to create the chandelier. You write a work order (Custom Resource request) specifying the design (properties). The artisan receives the order, builds the chandelier, and sends back a confirmation (response) with the finished product's serial number (physical resource ID). The crew then installs it. If you need to renovate (update) or demolish (delete) the house, the crew sends the artisan a change order or teardown request. The artisan must acknowledge each request and report completion; otherwise, the crew waits indefinitely (timeout). This is exactly how CloudFormation Custom Resources work: they extend the provisioning engine to handle any resource that isn't natively supported, using a Lambda-backed backend that must communicate back within a strict timeout.

## Core explanation

### What Are CloudFormation Custom Resources?

CloudFormation Custom Resources allow you to write custom provisioning logic for resources that are not natively supported by CloudFormation. They act as a bridge between CloudFormation and any AWS service or external system. The custom resource is defined in a CloudFormation template using the `AWS::CloudFormation::CustomResource` or `Custom::*` type. When CloudFormation creates, updates, or deletes the stack, it sends a request to a specified endpoint — typically an AWS Lambda function or an SNS topic — which performs the desired action and returns a response to CloudFormation.

### Why Custom Resources Exist

AWS CloudFormation supports hundreds of native resources, but you may need to:
- Provision resources from third-party services (e.g., a DNS record in Route53 that requires an API call not modeled as a native resource).
- Perform complex bootstrapping or configuration that isn't covered by `UserData` or `AWS::CloudFormation::Init`.
- Integrate with on-premises systems or other cloud providers.
- Implement custom validation or logic that depends on stack parameters.

Custom Resources fill this gap by allowing you to define arbitrary provisioning logic.

### How It Works Internally

The interaction between CloudFormation and the custom resource provider (Lambda) follows a strict protocol:

1. **CloudFormation sends a request** — When creating, updating, or deleting a stack, CloudFormation generates a JSON payload and sends it to the specified `ServiceToken`. The payload includes:
   - `RequestType`: `Create`, `Update`, or `Delete`.
   - `ResourceProperties`: The properties defined in the template.
   - `OldResourceProperties`: (only on Update) the previous properties.
   - `StackId`, `RequestId`, `ResourceType`, `LogicalResourceId`, `PhysicalResourceId` (for Update/Delete).

2. **Lambda function processes the request** — The Lambda function must perform the desired action (e.g., create an ACM certificate) and then send a response back to CloudFormation via a pre-signed S3 URL provided in the request (`ResponseURL`). The response must be a JSON object with:
   - `Status`: `SUCCESS` or `FAILED`.
   - `PhysicalResourceId`: A unique identifier for the resource (used in subsequent Update/Delete requests).
   - `StackId`, `RequestId`, `LogicalResourceId` (must match the request).
   - `Data`: Optional key-value pairs that can be referenced using `Fn::GetAtt` in the template.

3. **CloudFormation waits for the response** — CloudFormation waits up to **1 hour** (3600 seconds) for the response. If no response is received within that timeout, the stack operation fails and rolls back. The Lambda function has its own timeout (max 15 minutes), so the function must complete and send the response before either timeout expires.

### Key Components and Defaults

- **ServiceToken**: The ARN of the Lambda function or SNS topic. Must be in the same region.
- **Timeout**: CloudFormation's maximum wait time for a response is 1 hour. This is not configurable. However, you can set a shorter timeout using the `ServiceTimeout` property (in seconds) on the custom resource.
- **Lambda Timeout**: The Lambda function's timeout must be less than the CloudFormation timeout. Default Lambda timeout is 3 seconds; must be increased for custom resources (typically 5-10 minutes).
- **PhysicalResourceId**: Must be stable across updates; if it changes, CloudFormation treats it as a replacement (deletes old, creates new).
- **ResponseURL**: Pre-signed S3 URL valid for 1 hour. The Lambda function must PUT the response to this URL.

### Configuration and Verification

To create a custom resource, define it in the template:

```yaml
Resources:
  MyCustomResource:
    Type: Custom::MyCustomType
    Properties:
      ServiceToken: !GetAtt MyLambdaFunction.Arn
      Property1: value1
      Property2: value2
```

The Lambda function must include code to parse the request, perform the action, and send the response. A common pattern is to use the `cfn-response` module (Node.js) or `cfnresponse` (Python) to simplify sending responses.

Example Python handler:

```python
import cfnresponse
import boto3

def handler(event, context):
    physical_resource_id = event.get('PhysicalResourceId', None)
    try:
        if event['RequestType'] == 'Create':
            # Create resource
            physical_resource_id = 'unique-id'
        elif event['RequestType'] == 'Update':
            # Update resource
        elif event['RequestType'] == 'Delete':
            # Delete resource
        cfnresponse.send(event, context, cfnresponse.SUCCESS, {}, physical_resource_id)
    except Exception as e:
        cfnresponse.send(event, context, cfnresponse.FAILED, {}, physical_resource_id, reason=str(e))
```

**Important**: The function must send a response for every request type, including Delete. Failing to do so will cause the stack to hang and eventually fail.

### Interaction with Related Technologies

- **Lambda**: The most common service token. The Lambda function must have permissions to invoke the actions it performs (e.g., `acm:RequestCertificate`) and to PUT to the S3 response URL.
- **SNS**: You can use an SNS topic as the service token. The SNS topic triggers a Lambda function or sends to an HTTP endpoint. This adds decoupling but introduces an additional point of failure.
- **IAM**: The Lambda execution role must allow `logs:CreateLogGroup`, `logs:CreateLogStream`, `logs:PutLogEvents` for logging, and any resource-specific permissions.
- **CloudFormation Stack Drift**: Custom resources are not checked for drift by CloudFormation. You must implement your own drift detection logic if needed.

### Failure Handling and Rollback

If the Lambda function returns `FAILED` or times out, CloudFormation will:
- Mark the stack operation as failed.
- If the operation was a Create, roll back by deleting any resources created (including the custom resource, but the Lambda function must handle the Delete request during rollback).
- If the operation was an Update, roll back to the previous state. The Lambda function will receive a Delete request for the old resource and a Create request for the new resource (if the update caused replacement).

**Common mistake**: Not handling the Delete request during rollback. If the function does not respond to the Delete request, the stack will hang and eventually fail rollback.

### Best Practices

- Always send a response, even on failure.
- Use idempotent logic: the same Create request may be retried.
- Generate a unique `PhysicalResourceId` only on Create; reuse it on Update/Delete.
- Set appropriate Lambda timeout (e.g., 5 minutes) and CloudFormation timeout (e.g., 300 seconds) using `ServiceTimeout`.
- Log all requests and responses for debugging.
- Use the `cfn-response` library to avoid manual HTTP requests.

### Exam-Relevant Details

- The `ServiceToken` must be a Lambda ARN or SNS topic ARN.
- `ServiceTimeout` is an optional property that sets the maximum time (in seconds) CloudFormation waits for a response. Default is 3600 (1 hour).
- The Lambda function must respond to `Delete` requests even during stack rollback.
- If the Lambda function fails to respond, the stack will remain in `CREATE_IN_PROGRESS` or `UPDATE_IN_PROGRESS` until the timeout expires.
- Custom resources can return data via `Data` in the response, accessible with `Fn::GetAtt`.
- The `PhysicalResourceId` must be unique per resource; changing it triggers replacement.

## Real-world context

### Enterprise Scenario 1: Automated ACM Certificate Request

A company uses CloudFormation to deploy a multi-region application with HTTPS endpoints. ACM certificates are not natively supported by CloudFormation for DNS validation (only email validation was native at the time). They create a custom resource backed by a Lambda function that:
- On Create: Requests an ACM certificate with DNS validation, creates Route53 DNS records, and waits for validation.
- On Delete: Deletes the certificate and DNS records.
- The `PhysicalResourceId` is the certificate ARN.
- The function returns the certificate ARN in `Data`, which is used by the CloudFront distribution resource.

**Scale**: Deployed across 5 regions, each with 10 certificates. The Lambda function must handle concurrent requests and respect AWS API rate limits. **Pitfall**: If the Lambda function times out during DNS validation (which can take minutes), the stack hangs. They set `ServiceTimeout` to 600 seconds and Lambda timeout to 900 seconds.

### Enterprise Scenario 2: On-Premises Database Integration

A financial services company uses CloudFormation to provision EC2 instances that must be registered in an on-premises CMDB. They use a custom resource with a Lambda function that makes HTTP calls to the on-premises API via a VPN. The Lambda function:
- On Create: Registers the instance ID and IP in the CMDB.
- On Update: Updates the record.
- On Delete: Removes the record.
- Returns the CMDB record ID as `PhysicalResourceId`.

**Considerations**: The Lambda function must have VPC access to reach the on-premises API. Network latency and API failures must be handled with retries. **Misconfiguration**: If the Lambda function fails to respond due to a network timeout, CloudFormation will wait up to 1 hour before failing, causing a long deployment cycle.

### Enterprise Scenario 3: Custom AMI Creation Pipeline

A gaming company uses CloudFormation to build AMIs using a custom resource. The Lambda function triggers an EC2 Image Builder pipeline. The pipeline takes up to 30 minutes. The custom resource:
- On Create: Starts the Image Builder pipeline and polls until completion.
- On Delete: Does nothing (or deregisters the AMI).
- Returns the AMI ID in `Data`.

**Challenge**: The Lambda function must handle the long-running process. Since Lambda has a 15-minute max timeout, they cannot wait 30 minutes. Solution: Use an S3 bucket as a coordination point. The function starts the pipeline, sends a response immediately with a placeholder `PhysicalResourceId`, and then a separate process (e.g., EventBridge rule) updates the stack after completion. This is an advanced pattern called 'async custom resource'.

## Exam focus

### What DVA-C02 Tests on Custom Resources

The exam focuses on the Custom Resource lifecycle, the requirement to send a response, and the consequences of failing to do so. Objective 3.2 specifically includes 'Implement resource lifecycle hooks' and 'Troubleshoot stack failures'. Expect questions that test:
- The mandatory response to Delete requests.
- The effect of missing `PhysicalResourceId` or incorrect `Status`.
- The purpose of `ServiceToken` and `ServiceTimeout`.
- How `Fn::GetAtt` works with custom resources.

### Common Wrong Answers and Why Candidates Choose Them

1. **'The Lambda function does not need to respond to Delete requests if the resource was never created.'** — Wrong. CloudFormation always sends a Delete request, even if the Create failed. The function must respond to avoid hanging.
2. **'You can use any HTTP endpoint as the ServiceToken.'** — Wrong. The ServiceToken must be an ARN of a Lambda function or an SNS topic. Direct HTTP endpoints are not supported.
3. **'The Lambda function must complete within 1 hour.'** — Misleading. The Lambda function has its own timeout (max 15 min). The 1-hour is CloudFormation's timeout for receiving the response. The function must respond before both timeouts.
4. **'Custom resources can only be used in Create operations.'** — Wrong. They support Create, Update, and Delete.

### Specific Numbers and Terms That Appear on the Exam

- `ServiceToken`: Must be an ARN.
- `ServiceTimeout`: Default 3600 seconds (1 hour).
- `PhysicalResourceId`: Must be unique; if it changes on Update, CloudFormation treats it as replacement.
- `ResponseURL`: Pre-signed S3 URL.
- `cfn-response` module is used to send responses.

### Edge Cases and Exceptions

- **Stack rollback**: During rollback, CloudFormation sends Delete requests for resources that were created. The Lambda function must handle these.
- **No response**: If the Lambda function crashes without sending a response, the stack will remain in progress until the timeout.
- **Update with replacement**: If the `PhysicalResourceId` changes during an Update, CloudFormation first sends a Delete for the old resource, then a Create for the new one.

### How to Eliminate Wrong Answers

- If an answer says 'The Lambda function only needs to respond to Create requests', eliminate it — all request types need a response.
- If an answer says 'Use an HTTP URL as ServiceToken', eliminate it — must be ARN.
- If an answer says 'Custom resources are checked for drift', eliminate it — they are not.
- If an answer mentions 'ServiceTimeout' as a Lambda property, eliminate it — it's a CloudFormation resource property.

## Step by step

1. **Define Custom Resource in Template** — In your CloudFormation template, declare a resource with type `Custom::MyType` or `AWS::CloudFormation::CustomResource`. Provide a `ServiceToken` property pointing to the ARN of the Lambda function that will handle the provisioning. You can also pass additional properties that the Lambda function will receive in the `ResourceProperties` object. CloudFormation will generate a unique `RequestId` and `StackId` for each request.
2. **CloudFormation Sends Create Request** — When the stack is created, CloudFormation constructs a JSON request with `RequestType` set to `'Create'`, `ResourceProperties` containing the properties from the template, and a `ResponseURL` (a pre-signed S3 URL). This request is sent to the `ServiceToken` (Lambda). The Lambda function is invoked synchronously with this event. CloudFormation then waits for a response to the `ResponseURL`.
3. **Lambda Processes Request and Responds** — The Lambda function parses the event, performs the desired action (e.g., create an ACM certificate), and generates a response. The response must include `Status` (`'SUCCESS'` or `'FAILED'`), `PhysicalResourceId` (a unique identifier for the resource), and optionally `Data` (key-value pairs). The function sends this response via an HTTP PUT to the `ResponseURL`. It must do so before the Lambda timeout (up to 15 min) and the CloudFormation timeout (default 1 hour).
4. **CloudFormation Receives Response and Continues** — CloudFormation receives the HTTP PUT at the pre-signed URL. If `Status` is `'SUCCESS'`, it continues stack creation. The `PhysicalResourceId` is stored in the stack metadata. If `Status` is `'FAILED'`, CloudFormation marks the resource as failed and begins stack rollback. During rollback, CloudFormation sends a `Delete` request to the same Lambda function for the custom resource.
5. **Update and Delete Operations** — On stack update, CloudFormation sends an `Update` request with both `ResourceProperties` and `OldResourceProperties`. The Lambda function must update the resource accordingly and send a response. On stack deletion (or rollback), CloudFormation sends a `Delete` request. The Lambda function must clean up the resource and respond. If no response is received within the timeout, the stack operation fails.

## Comparisons

### Custom Resource with Lambda vs Custom Resource with SNS

**Custom Resource with Lambda:**
- Direct invocation of Lambda; simpler setup.
- Response is sent directly from Lambda to CloudFormation.
- Lambda timeout limits execution to 15 minutes.
- Easier to debug via CloudWatch Logs.
- Commonly tested on DVA-C02.

**Custom Resource with SNS:**
- SNS can fan out to multiple subscribers (e.g., Lambda + HTTP).
- Response must be sent by the subscriber; SNS does not respond automatically.
- No built-in timeout; relies on subscriber.
- Adds decoupling but increases complexity.
- Less common on the exam, but possible.

## Common misconceptions

- **Misconception:** Custom resources can only be used with Lambda functions. **Reality:** The ServiceToken can be a Lambda function ARN or an SNS topic ARN. SNS can fan out to multiple endpoints, including Lambda, HTTP, or email.
- **Misconception:** The Lambda function must respond within 15 minutes (Lambda timeout). **Reality:** Lambda timeout is 15 minutes max, but CloudFormation's ServiceTimeout defaults to 1 hour. The function must respond before either timeout. You can set a shorter ServiceTimeout to fail fast.
- **Misconception:** If the Lambda function fails during Create, the Delete request is not sent. **Reality:** CloudFormation always sends a Delete request during rollback, even if the Create failed. The function must handle it and respond.
- **Misconception:** You can use Fn::GetAtt to retrieve any attribute from a custom resource. **Reality:** Fn::GetAtt only retrieves attributes that were returned in the Data map of the response. If the Lambda function does not return a Data key, Fn::GetAtt will fail.
- **Misconception:** Custom resources support only Create and Delete operations. **Reality:** Custom resources support Create, Update, and Delete. The Lambda function must handle all three RequestTypes.

## Key takeaways

- Custom Resources extend CloudFormation to provision non-native resources using Lambda or SNS as the service token.
- The Lambda function must send a response (SUCCESS or FAILED) to the pre-signed S3 URL for every request type: Create, Update, and Delete.
- CloudFormation waits up to 1 hour for a response; the Lambda function must respond within both its own timeout (max 15 min) and the CloudFormation timeout.
- If no response is received, the stack operation hangs and eventually fails with a timeout error.
- The PhysicalResourceId must be stable; changing it on Update triggers resource replacement (Delete old, Create new).
- Custom resources are not checked for drift; you must implement custom drift detection if needed.
- The ServiceToken must be an ARN (Lambda or SNS); direct HTTP endpoints are not supported.
- Use the cfn-response module (Node.js/Python) to simplify sending responses and avoid manual HTTP requests.
- Always handle Delete requests, even during stack rollback, to avoid stack hang.
- Custom resources can return data via the Data field, accessible with Fn::GetAtt in the template.

## FAQ

**What happens if my Lambda function does not send a response to CloudFormation?**

If the Lambda function does not send a response (either because it crashed or the code didn't call the response URL), CloudFormation will wait for the duration of the ServiceTimeout (default 1 hour). After that, the stack operation fails with a timeout error. The stack will be rolled back, and during rollback, CloudFormation will send a Delete request to the same function. If the function still does not respond, the rollback will also hang. Always ensure your function sends a response in all code paths, including error handlers.

**Can I use a custom resource to create a resource that takes longer than 15 minutes?**

The Lambda function has a maximum timeout of 15 minutes. If your provisioning action takes longer, you cannot wait synchronously in Lambda. Instead, use an asynchronous pattern: have the Lambda function start the long-running process, return immediately with a placeholder PhysicalResourceId, and then have a separate mechanism (e.g., EventBridge rule, Step Functions) update the stack with the actual resource ID using the CloudFormation UpdateStack API. This is an advanced pattern and is not commonly tested on DVA-C02.

**What is the difference between ServiceTimeout and Lambda timeout?**

ServiceTimeout is a property of the custom resource in the CloudFormation template. It specifies the maximum time (in seconds) that CloudFormation will wait for a response from the custom resource provider. The default is 3600 (1 hour). Lambda timeout is a property of the Lambda function itself, with a maximum of 900 seconds (15 minutes). The effective timeout for the custom resource operation is the minimum of these two values. You should set ServiceTimeout to a value slightly higher than the expected Lambda execution time to avoid premature failures.

**Can I use Fn::GetAtt to get the PhysicalResourceId of a custom resource?**

No, the PhysicalResourceId is not accessible via Fn::GetAtt. It is used internally by CloudFormation to track the resource. To expose data to other parts of the template, your Lambda function must return key-value pairs in the Data field of the response. For example, if you return {"Data": {"Arn": "arn:..."}}, you can use !GetAtt MyCustomResource.Arn in the template.

**What happens if I change the ServiceToken after stack creation?**

If you update the ServiceToken property (e.g., change the Lambda function ARN), CloudFormation will treat the custom resource as requiring an update. It will send an Update request to the new ServiceToken. However, the PhysicalResourceId should remain the same to avoid replacement. If the PhysicalResourceId changes, CloudFormation will delete the old resource and create a new one.

**Is it possible to test a custom resource without creating a stack?**

Yes, you can manually invoke the Lambda function with a test event that mimics the CloudFormation request. You must provide a valid ResponseURL (you can generate a pre-signed S3 URL or use a dummy URL for testing, but the function will try to PUT to it). A better approach is to use the AWS CloudFormation console's 'Test' button (if available) or use the AWS CLI with the `aws cloudformation test-custom-resource` command (not a real command; you can simulate by invoking Lambda directly).

**Can a custom resource have dependencies on other resources?**

Yes, you can use DependsOn to ensure that other resources are created before the custom resource. Also, you can reference other resources using Ref or Fn::GetAtt in the Properties of the custom resource. CloudFormation will pass those resolved values in the ResourceProperties to the Lambda function.

---

Interactive version with quiz and diagrams: https://courseiva.com/learn/dva-c02/cfn-custom-resources