Error Handling Patterns in Serverless Apps

Serverless applications, by their nature, involve many moving parts: Lambda functions, event sources (S3, SQS, API Gateway), and downstream services (DynamoDB, SNS). Failures can occur at any layer: code exceptions, timeouts, throttling, permission errors, or downstream service unavailability. Error handling patterns are the structured approaches to detect, capture, and respond to these failures without losing data or causing cascading failures.

On the DVA-C02 exam, you must understand how Lambda handles errors differently based on invocation type (synchronous vs. asynchronous) and event source mapping (e.g., SQS, DynamoDB Streams). The exam also tests your ability to configure DLQs (Dead Letter Queues), set retry policies, and use Lambda destinations for failure routing.

Lambda supports three invocation types: - Synchronous (e.g., API Gateway, SDK Invoke with InvocationType='RequestResponse'): The caller receives the error response immediately. Lambda does not retry. The calling service (e.g., API Gateway) may have its own retry logic. - Asynchronous (e.g., S3 events, SNS, SDK Invoke with InvocationType='Event'): Lambda automatically retries the failed invocation twice (total of 3 attempts) with delays between attempts (approximately 1 minute then 2 minutes, but not guaranteed). After all retries fail, the event can be sent to a Dead Letter Queue (DLQ) or a Lambda destination (on failure). - Poll-based (Event Source Mappings) (e.g., SQS, DynamoDB Streams, Kinesis): The event source mapping polls the source and invokes Lambda synchronously. If the function fails, the mapping retries based on the source's configuration (e.g., SQS visibility timeout, Kinesis iterator age). Successful processing deletes the message; failure causes it to reappear for reprocessing.

You can configure DLQs and destinations via the AWS Console, CLI, or CloudFormation. For example, to set a DLQ for async invocations:

aws lambda put-function-event-invoke-config \
  --function-name my-function \
  --destination-config '{"OnFailure": {"Destination": "arn:aws:sqs:us-east-1:123456789012:my-dlq"}}'

For SQS event source mapping, the DLQ is configured on the SQS queue itself (redrive policy):

aws sqs set-queue-attributes \
  --queue-url https://sqs.us-east-1.amazonaws.com/123456789012/my-queue \
  --attributes '{"RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:us-east-1:123456789012:my-dlq\",\"maxReceiveCount\":\"5\"}"}'

To verify, check CloudWatch Logs for error messages like Task timed out after 3.00 seconds or Process exited before completing request. Also monitor the DeadLetterErrors metric (DLQ send failures) and DestinationDeliveryFailures.

Lambda function timeout defaults to 3 seconds (max 15 minutes). If your function exceeds the timeout, it is terminated and returns an error. For async invocations, this triggers a retry. For synchronous, the caller gets a Task timed out error.

Throttling: Lambda has a concurrency limit (default 1000 per region). If exceeded, synchronous invocations return TooManyRequestsException (429). Asynchronous invocations are queued and retried automatically for up to 6 hours. Exam tip: Throttled async invocations do not go to DLQ unless the maximum-event-age expires.

Poison messages are messages that repeatedly fail processing. Without a DLQ, they would be retried indefinitely (or until the visibility timeout expires). DLQs isolate these messages for manual inspection. Idempotency ensures that reprocessing the same message does not cause side effects (e.g., duplicate charges). Use idempotency tokens (e.g., DynamoDB table with TTL) to deduplicate.

In a production e-commerce platform, an order processing Lambda function receives events from SQS. The function writes to DynamoDB and calls an external payment gateway. If the payment gateway times out, the function returns an error. Without a DLQ, the same message would be retried every 30 seconds (visibility timeout) until it exceeds the max receive count (set to 3). After 3 failures, it moves to the DLQ. The operations team monitors the DLQ for failed orders and manually reprocesses them after fixing the issue (e.g., payment gateway down). They also set up a CloudWatch alarm on the DLQ's ApproximateNumberOfMessagesVisible to trigger an SNS notification to the on-call engineer.

Another scenario: a media processing pipeline uses Lambda to transcode videos uploaded to S3. The Lambda function has a 5-minute timeout. If a video is too large, the function times out. With async invocation, Lambda retries twice, but each retry also takes 5 minutes, causing a 15-minute delay before the event goes to the DLQ. To improve, the team sets maximum-event-age-in-seconds to 300 (5 minutes) so that events older than 5 minutes are immediately sent to the DLQ without retries. They also use Lambda destinations to send the failure payload to an SNS topic that triggers a notification to the engineering team.

A common misconfiguration is setting the SQS visibility timeout lower than the Lambda function timeout. For example, if Lambda timeout is 30 seconds but visibility timeout is 20 seconds, the message becomes visible again before the function finishes, leading to duplicate processing. The rule of thumb: set visibility timeout to 6x the Lambda timeout (to allow for retries and DLQ processing). In practice, for a 30-second Lambda, set visibility timeout to 180 seconds.

Another issue is forgetting to enable ReportBatchItemFailures for SQS event source mappings. Without this, a single failing record causes the entire batch to be reprocessed, even if other records succeeded. Enabling this feature allows Lambda to return partial failures, so only the failed records are retried. This is a key exam topic.