Lambda Destinations and Async Invocations

Lambda functions can be invoked synchronously (InvocationType='RequestResponse') or asynchronously (InvocationType='Event'). For async invocations, Lambda queues the event and returns a 202 response immediately. The function processes the event later. This is ideal for non-latency-sensitive workloads like processing S3 events, SNS notifications, or scheduled tasks.

Async invocations decouple the producer from the consumer. The producer (e.g., S3) doesn't wait for the function to complete. This improves scalability and fault tolerance. If the function fails, Lambda automatically retries twice (total 3 attempts) with a 1-minute wait between retries. After all retries fail, the event can be sent to a Dead Letter Queue (DLQ) or a Lambda Destination (on failure).

When you invoke a Lambda function asynchronously, the event is placed in an internal SQS queue managed by Lambda. Lambda's internal poller reads from this queue and invokes your function. If the function returns an error (including throttling or invocation errors), Lambda retries based on the configured retry policy. The default retry count is 2 (so 3 total attempts). The maximum event age is 6 hours (default) but can be set from 60 seconds to 6 hours. After the event expires or all retries fail, the event is discarded unless a DLQ or Destination is configured.

Lambda Destinations allow you to route the result of an async invocation to another AWS service without writing code. You can configure destinations for two states:

Each destination can be one of:

You can configure up to two destinations per function (one for success, one for failure). Destinations receive a JSON payload containing the result, request context, and response context.

Destinations are configured on a function's asynchronous invocation configuration. You can do this via AWS Console, CLI, or SDK. CLI example:

aws lambda put-function-event-invoke-config \
  --function-name my-function \
  --destination-config '{"OnSuccess":{"Destination":"arn:aws:sqs:us-east-1:123456789012:my-queue"},"OnFailure":{"Destination":"arn:aws:sns:us-east-1:123456789012:my-topic"}}'

You can also configure event invoke config for a specific qualifier (version or alias) using the --qualifier parameter.

Before Destinations, the only way to capture failed events was a Dead Letter Queue (DLQ). DLQ can be an SQS queue or SNS topic. DLQ receives events only after all retries are exhausted. Destinations are more flexible because they can handle both success and failure, and they provide richer context. However, DLQ and Destinations are not mutually exclusive; you can have both, but if both are configured, the Destination takes precedence for failure events (the event is sent to the Destination, not the DLQ).

Many AWS services invoke Lambda asynchronously: S3, SNS, CloudWatch Events, EventBridge, etc. When these services invoke Lambda, they use async invocation by default. The function's async configuration (including Destinations) applies. For example, if S3 sends an event to Lambda and the function fails, after retries, the event can be sent to an SNS topic via Destination.

The destination receives a JSON payload with the following structure:

{
  "version": "1.0",
  "timestamp": "2019-11-24T23:08:25.651Z",
  "requestContext": {
    "requestId": "c2a6f2ae-...",
    "functionArn": "arn:aws:lambda:us-east-1:123456789012:function:my-function",
    "condition": "Success",
    "approximateInvokeCount": 1
  },
  "requestPayload": {
    ... // original event
  },
  "responseContext": {
    "statusCode": 200,
    "executedVersion": "$LATEST",
    "functionError": "" // only on failure
  },
  "responsePayload": {
    ... // function's return value or error object
  }
}

If you set retry attempts to 0, the function is invoked once; if it fails, it goes to Destination/DLQ immediately.

When a function fails asynchronously, Lambda captures the error and retries. If the function is throttled (concurrency limit), Lambda retries with backoff. If the function code throws an unhandled exception, it counts as a failure. If the function times out (max 15 min), it also counts as a failure.

You can monitor async invocations via CloudWatch metrics: AsyncEventsReceived, AsyncEventAge, AsyncEventsDropped. For Destinations, you can log the destination invocation via CloudWatch Logs if the destination is a Lambda function.

A media company uploads images to S3. Each upload triggers a Lambda function that resizes the image and stores it in another S3 bucket. This is an async invocation (S3 -> Lambda). They need to know if processing succeeded or failed. They configure a Destination on success to an SNS topic that sends an email to the operations team. On failure, they send the event to an SQS queue for manual reprocessing. They set max event age to 2 hours and retry attempts to 1 (total 2). This ensures that transient failures are retried once, but if it fails again, it goes to the queue. In production, they handle thousands of uploads per minute. They monitor AsyncEventsDropped metric to catch any events that expired. They also set a concurrency limit to avoid overwhelming downstream services.

An e-commerce platform uses EventBridge to trigger a Lambda function for order validation. The function validates the order and updates a database. They use async invocation because the validation is not time-critical. They configure a Destination on failure to another Lambda that sends a notification to the customer service team via Slack (using webhook). They also configure a DLQ (SQS) as a backup. However, they realize that if both Destination and DLQ are configured, the Destination takes precedence, so the DLQ is never used. They remove the DLQ to avoid confusion. They set retry attempts to 2 (total 3) and max event age to 5 minutes because orders should be processed quickly. They also use SQS FIFO as the Destination for failure to maintain order of failed events.