Lambda Architecture Patterns

What is Lambda Architecture Patterns?

Lambda architecture patterns refer to the design approaches for building event-driven, serverless applications using AWS Lambda. At its core, Lambda is a compute service that runs code in response to events and automatically manages the underlying compute resources. The patterns dictate how functions are triggered, how they communicate, how they scale, and how they integrate with other AWS services. The exam tests your ability to choose the right pattern for a given requirement, such as minimizing latency, handling high throughput, or ensuring idempotency.

Synchronous vs. Asynchronous Invocation

Lambda supports three invocation types: synchronous (RequestResponse), asynchronous (Event), and poll-based (from services like SQS or DynamoDB Streams). In synchronous invocation, the caller waits for the function to complete and returns a response. Common triggers: API Gateway, ALB, Cognito, and Step Functions. Use this when you need a real-time response, but beware of timeout limits (max 15 minutes) and cold start latency. Asynchronous invocation queues the event and returns immediately. Lambda automatically retries twice (total 3 attempts) with a backoff up to 6 hours. Use this for non-urgent tasks like data processing or notifications. Poll-based invocation occurs when Lambda pulls from a stream (Kinesis, DynamoDB) or queue (SQS). Lambda manages the polling and batch processing. The exam often asks: "Which invocation type should you use for a web API?" Answer: synchronous via API Gateway. "For processing uploaded files?" Answer: asynchronous, triggered by S3 events.

Cold Starts and Provisioned Concurrency

Cold start occurs when Lambda initializes a new execution environment, adding latency (typically 100ms-2s for Java/.NET, 50-500ms for Python/Node). Provisioned Concurrency keeps a specified number of environments warm, eliminating cold starts. It is billed per second regardless of invocations. Use it for latency-sensitive functions (e.g., user-facing APIs). The exam expects you to know that Provisioned Concurrency can be scheduled using Application Auto Scaling or configured with a reserved concurrency minimum. Another trick: if a function has a VPC configuration, cold starts are longer due to ENI setup. To mitigate, use RDS Proxy for database connections or keep functions outside VPC if possible.

Function Orchestration: Step Functions vs. Direct Chaining

For complex workflows, you can chain Lambda functions by having one function invoke another, but this creates tight coupling and makes error handling difficult. Step Functions is a state machine service that orchestrates multiple Lambda functions with built-in error handling, retries, and parallel execution. The exam favors Step Functions over direct chaining for multi-step processes. Example: a video processing pipeline: Step Functions triggers a Lambda to transcode, then another to generate thumbnails, then another to update the database. If a step fails, Step Functions can retry or route to a fallback. Direct chaining is acceptable for simple linear sequences but not for branching or long-running workflows.

Event Filtering and Batching

Lambda can filter events before processing. For SQS, you can set a filter policy to only consume messages with certain attributes. For Kinesis and DynamoDB Streams, you can configure a filter pattern (e.g., only process records where eventName equals 'INSERT'). This reduces invocations and cost. Batching allows you to process multiple records in a single invocation. For SQS, you set batchSize (max 10) and maximumBatchingWindowInSeconds (max 300). For Kinesis, batchSize (max 10,000) and batchWindow (max 300). The exam tests: when should you use batching? When you want to reduce number of invocations and cost, but be aware that latency increases. Also, for SQS, if you set batchSize to 10, but only 3 messages arrive, Lambda will still invoke with those 3 after the batching window expires.

Error Handling and Dead Letter Queues (DLQ)

For asynchronous invocation, Lambda automatically retries twice. If all attempts fail, the event can be sent to a DLQ (SQS or SNS). This is configured on the function or event source mapping. For synchronous invocation, you must handle errors in the caller (e.g., API Gateway returns 500). For poll-based invocations, failed records can be sent to a DLQ after exhausting retries (e.g., SQS redrive policy). The exam often presents a scenario where you need to capture failed events for analysis – answer: configure a DLQ. Also note: for Kinesis, you can set bisectBatchOnFunctionError to split the batch on failure and retry smaller batches.

Resource Configuration: Memory, Timeout, and Concurrency

Lambda memory can be set from 128 MB to 10,240 MB (in 1 MB increments). CPU and network throughput scale proportionally with memory. Timeout max is 15 minutes (900 seconds). Concurrency limits: by default, 1000 concurrent executions per region (soft limit, can be increased). Reserved concurrency guarantees a set number of concurrent executions for a function, preventing other functions from using that capacity. Provisioned Concurrency is separate. The exam tests: if you need to reduce latency for a CPU-intensive function, increase memory (which also increases CPU). If you need to limit a function's impact on downstream resources, use reserved concurrency.

Lambda@Edge and CloudFront Functions

Lambda@Edge runs Lambda functions at CloudFront edge locations, allowing you to modify requests/responses with low latency. There are four event triggers: viewer request, origin request, origin response, viewer response. Lambda@Edge has restrictions: max 5-second timeout for viewer events, 30 seconds for origin events, no VPC support, and environment variables are limited. CloudFront Functions is a lighter alternative for simple JavaScript operations (e.g., URL redirects) with sub-ms latency. The exam distinguishes: use Lambda@Edge for complex logic (e.g., authentication, A/B testing) and CloudFront Functions for simple header manipulation.

Security: IAM Roles and Resource Policies

Lambda uses an execution role (IAM role) to access other AWS services. The role must have policies granting permissions (e.g., DynamoDB read, S3 write). For cross-account access, you use resource-based policies on the Lambda function to allow other accounts to invoke it. The exam tests: how to allow an S3 bucket in another account to invoke a Lambda function? Answer: add a resource-based policy to the Lambda function allowing the S3 service principal from the other account. Also, remember that Lambda functions in a VPC need a VPC endpoint for services like DynamoDB or S3 to avoid NAT gateway costs.

Performance Optimization: Connection Pooling and Caching

Lambda functions should reuse database connections across invocations by declaring them outside the handler. This reduces latency and load on the database. For example, in Python, create a DynamoDB client at global scope. Also, use ElastiCache or DAX for caching frequent reads. The exam expects you to know that static assets should be stored in S3 with CloudFront, not in Lambda. For large files, use S3 presigned URLs.

Monitoring and Logging

Lambda automatically sends logs to CloudWatch Logs. You can also emit custom metrics via CloudWatch Embedded Metric Format. The exam asks: how to monitor invocation errors? Answer: CloudWatch metrics (Invocations, Errors, Throttles) and CloudWatch Logs. For distributed tracing, use AWS X-Ray. Enable active tracing in Lambda configuration to see traces for each invocation.

Best Practices for the Exam

Common Pitfalls

Integration with Other Services

Summary of Key Exam Values

Enterprise Scenario 1: E-commerce Order Processing

A large online retailer uses Lambda to process orders. The flow: API Gateway invokes a Lambda synchronously to validate the order and return a confirmation. Then, an SQS message is sent to trigger a second Lambda asynchronously for payment processing, inventory update, and email notification. The payment processing Lambda uses Provisioned Concurrency to ensure low latency. The inventory update Lambda uses DynamoDB Streams to trigger a third Lambda that updates a search index. Challenges: the payment Lambda must be idempotent to handle retries; a DLQ captures failed payments for manual review. Performance: at peak (1000 orders/second), the SQS queue buffers spikes, and Lambda scales to 500 concurrent executions. Misconfiguration example: setting the SQS visibility timeout too low causes messages to reappear before processing completes, leading to duplicate payments. The solution: set visibility timeout to 6x function timeout (e.g., 30 seconds for a 5-second function).

Enterprise Scenario 2: Real-Time Data Analytics

A financial services company ingests stock market data from Kinesis Data Streams. A Lambda function processes records in near real-time (poll-based). The function aggregates prices and writes to DynamoDB for a dashboard. To handle high throughput (10,000 records/second), they use batch size of 500 and a batch window of 10 seconds. They enable bisectBatchOnFunctionError to isolate bad records. Provisioned Concurrency is not needed because latency tolerance is ~5 seconds. The Lambda function uses a VPC to access an RDS database for enrichment; to reduce cold starts, they use RDS Proxy. Misconfiguration: setting the batch size too large (10,000) without adjusting memory leads to timeouts. The fix: increase memory (and CPU) to process larger batches faster. Also, they use a DLQ on Kinesis to capture failed records for replay.

Enterprise Scenario 3: Serverless Web Application Backend

A startup builds a serverless web app using Lambda, API Gateway, and DynamoDB. The Lambda functions are behind API Gateway with caching enabled. They use CloudFront for CDN. For user authentication, Cognito triggers Lambda during sign-up to validate user data. They use Step Functions to orchestrate a multi-step onboarding process (validate, create user, send welcome email). Cold starts are mitigated by using Provisioned Concurrency for the main API functions. They also use Lambda layers for shared business logic. Misconfiguration: not setting a DLQ for async invocations leads to silent failures when sending emails. The fix: configure a DLQ on the function. Another issue: the Step Functions workflow times out because a Lambda function exceeds the 15-minute limit. The solution: move long-running tasks to ECS Fargate or break the workflow into smaller steps.