Design Resilient Architectures practice questions

Change the Aurora cluster to Single-AZ to reduce failover events.

Why it fails: Changing the Aurora cluster from Multi-AZ to Single-AZ does not reduce the frequency of failover events; in fact, it increases the blast radius when an AZ failure or instance health issue occurs. Aurora storage is already replicated across three AZs, but a Single-AZ configuration means the primary DB instance has no cross-AZ standby or replacement target to promote during a failover, so availability is degraded rather than improved. This option trades away the very redundancy that allows Aurora to recover quickly, making connection disruption worse rather than smoothing it over.

Increase the application thread count so more requests can be served while connections reconnect.

Why it fails: Raising the application thread count during a connection disruption amplifies connection churn because every new thread that needs a JDBC connection will attempt to obtain one from a pool already filled with stale, broken connections from the old writer. This creates a thundering herd of retry requests that can overwhelm the newly promoted writer instance, increasing CPU and memory pressure and slowing transaction recovery. Pooled connections are not refreshed by adding threads; the application still needs a mechanism like RDS Proxy to transparently replace backend connections and reduce the stall and retry storm.

Pin all database traffic to a specific instance hostname instead of the writer cluster endpoint.

Why it fails: Pinning database traffic to a specific instance hostname, such as the old writer's instance endpoint, defeats the purpose of the Aurora cluster endpoint because the cluster endpoint uses DNS to automatically redirect applications to the new writer after a failover. The instance endpoint remains associated with the same physical node, which becomes read-only or unavailable after a role switch, so writes fail with read-only or connection errors. This approach directly undermines Aurora's failover design, making an already disruptive failover into an application outage.

Backup and restore only, relying on scheduled snapshots and manual restores during incidents.

Why it fails: Backup/restore can meet data-loss tolerance in some cases, but RTO of 1 hour is often too aggressive for manual restores.

Pilot light, keeping only minimal infrastructure in the secondary Region and starting full services after failover.

Why it fails: Pilot light is cheaper, but RTO of 1 hour may not be achievable if full compute/services must be brought up from near-zero.

Active/active, routing production traffic to both Regions continuously and accepting dual-region complexity.

Why it fails: Active/active best fits very low RTO and high availability, but it usually costs more than the stated budget constraints.

Application Load Balancer (ALB): Automatically replaces failed instances across Availability Zones.

Why it fails: The claim that an Application Load Balancer automatically replaces failed instances is incorrect because the ALB is a traffic distribution device, not a compute capacity manager. When an instance fails or becomes unhealthy, the ALB detects it via health checks and removes it from the target group, but it cannot launch new instances; that action belongs to an Auto Scaling group. Mixing these responsibilities is a common misunderstanding of AWS's separation between load balancing and instance lifecycle management.

Auto Scaling group: Distributes incoming traffic across healthy instances in multiple Availability Zones.

Why it fails: An Auto Scaling group does not distribute incoming traffic across healthy instances; its role is to manage the number and placement of instances, launching and terminating them to match a desired capacity or scaling policy. The distribution of traffic to healthy instances is performed by an Application Load Balancer or other load balancer, which accepts requests and forwards them to the ASG-managed targets. Conflating the ASG's capacity management function with the ALB's traffic routing function is incorrect for this architecture.

Create a read replica in the same Availability Zone as the primary database.

Why it fails: A read replica in the same Availability Zone shares the primary's failure domain, so an AZ outage removes both instances and no failover occurs. Read replicas serve read scaling and offload analytical queries; they become relevant when the requirement is distributing read traffic, not automatic AZ-level failover.

Increase the backup retention period to 35 days.

Why it fails: Increasing backup retention to 35 days only extends how far back you can restore; it provides no automatic failover when an Availability Zone fails. It is tempting because retention settings genuinely support point-in-time recovery and compliance archiving, but that addresses data restoration after an incident, not the requirement for continuous availability during an AZ outage.

Move the database to an EC2 instance with an attached EBS volume.

Why it fails: A single EC2 instance with an EBS volume is tied to one Availability Zone, so an AZ failure takes the database down with no automatic failover. This suits standalone workloads where resilience is handled at the application tier.

Create a single Route 53 record using weighted routing across both ALBs with weights adjusted manually during an incident.

Why it fails: Weighted routing does not automatically detect health conditions and shift traffic based on impairment without manual intervention.

Use latency-based routing so Route 53 always selects the fastest region; health checks are unnecessary because client latency reflects availability.

Why it fails: Latency-based routing selects based on performance, not health, and does not guarantee failover when the primary region is unhealthy.

Use a single A record with a static IP address that points to a NAT gateway, and update that IP during failure events.

Why it fails: Static IPs and NAT gateways are not an appropriate mechanism for regional ALB failover, and updating during incidents is manual.

Pilot light: replicate databases and store backups, keep only minimal infrastructure in the secondary Region, and scale up fully during failover.

Why it fails: Pilot light can work for some RTO goals, but fully scaling dependencies within 2 hours may be difficult depending on application initialization and data warm-up.

Backup and restore only: rely on daily automated backups and restore into the secondary Region during an incident.

Why it fails: Daily backups cannot meet a 15-minute RPO, and restoration time would likely exceed the 2-hour RTO.

Multi-site active-active: run both Regions at full capacity and route live traffic to both simultaneously.

Why it fails: Active-active typically costs more than allowed, and it adds complexity beyond what the budget and requirements describe.

Switch from RDS Single-AZ to RDS Multi-AZ, keeping the EC2 instances in only AZ-a because failover will still reach them.

Why it fails: Switching RDS from Single-AZ to Multi-AZ only protects the database by maintaining a synchronously replicated standby in a second Availability Zone; it does nothing to increase the number or distribution of EC2 compute instances behind the ALB. If AZ-a is impaired, both the primary and any standby remain accessible only from instances that are themselves in the degraded AZ, so the web/application tier still cannot process checkout requests. The database is not the bottleneck, and a database failover does not create the missing healthy application capacity needed to absorb the checkout delay.

Terminate the ALB and use a Network Load Balancer (NLB) in front of the existing single-AZ EC2 instances.

Why it fails: Replacing the ALB with a Network Load Balancer does not change the fact that all EC2 instances live only in AZ-a, so when that Availability Zone is degraded the NLB has no healthy targets in another AZ to fail over to. An NLB operates at Layer 4 and does not perform HTTP path-based routing or deep health checks, yet it still depends entirely on the health and reachability of the registered targets. The single-AZ target group is the root limitation, and no load balancer type can introduce compute capacity into a second Availability Zone. The ALB is already appropriate for an HTTP-based ecommerce app; the missing component is cross-AZ auto scaling.

Add more EC2 instances in AZ-a and increase the ALB health check thresholds to avoid unnecessary replacements during impairments.

Why it fails: Piling additional EC2 instances into AZ-a only raises the total amount of compute that will be affected when that single Availability Zone suffers an impairment, because all instances share the same physical infrastructure and network path. Raising the ALB health check thresholds makes the load balancer more tolerant of failing health checks, which delays marking unhealthy instances as out-of-service and prolongs the period during which the ALB routes requests to degraded targets. This increases latency and error rates rather than avoiding replacements; health check thresholds do not fix an underlying lack of cross-AZ failure domain. The correct fix is to place instances in multiple AZs so the ALB can route away from the impaired zone entirely.

Use a single Route 53 A record that points only to Region A’s ALB and manually update it after failures.

Why it fails: A single A record that only references Region A's ALB provides no health-check-driven failover; if that ALB becomes unhealthy, Route 53 continues returning the same record until you manually edit it. DNS resolvers cache the A record for the configured TTL, so even after your change, clients may still hit Region A for the entire TTL period. This manual process violates the requirement for automatic recovery and introduces both delay and human error.

Use Route 53 latency-based routing with separate records for each region.

Why it fails: Latency-based routing selects the AWS region with the lowest network latency for each individual user, not the region that is healthiest. If Region A is down, Route 53 still returns Region A's record for users who measure it as lowest latency, because no health check is associated to mark it unhealthy. Therefore, rather than providing automatic failover, latency routing can actually continue sending traffic to a failed Region A and increase downtime.

Use weighted routing and set the Region B weight to 0 to ensure it is only used when needed.

Why it fails: Setting Region B's weight to 0 in weighted routing means that record is never returned as long as Region A's record has any positive weight, so it cannot absorb traffic when Region A fails. Weighted routing simply distributes DNS responses according to static proportions; it does not monitor endpoint health or shift weights automatically in response to outages. To make weighted records fail over you would need to manually adjust weights and rely on health checks, and even then it is not a substitute for Route 53's purpose-built failover policy.

Enable S3 lifecycle policies to automatically move objects to Glacier after 30 days only.

Why it fails: Lifecycle policies only transition objects to Glacier within the same bucket; they neither prevent overwrites nor replicate data to another Region. It is tempting as a cost-tiering measure, but the scenario needs versioning for overwrite protection and cross-Region replication for the DR copy.

Disable all versioning and rely on AWS Backup to restore objects from a scheduled backup window.

Why it fails: Disabling versioning removes the only S3 mechanism that preserves prior object versions, so an accidental overwrite destroys the original outright; AWS Backup restores only to its last scheduled window, losing intervening writes. AWS Backup suits compliance-driven, schedule-based recovery, not continuous overwrite protection plus cross-Region copies.

Enable S3 Block Public Access and SSE-S3 encryption, without using versioning or replication.

Why it fails: Block Public Access and SSE-S3 govern access and encryption at rest; neither retains prior object versions nor copies data to a second Region, so overwrites remain unrecoverable and no DR copy exists. These controls are the right answer when the requirement is locking down exposure and encrypting stored objects.

Pilot light: replicate databases and store backups, keep only minimal infrastructure in the secondary Region, and scale up fully during failover.

Why it fails: In a pilot light DR configuration, only the database and storage layers are kept active in the secondary Region; the application compute is stopped or not deployed. During failover, you must launch EC2 instances, attach Auto Scaling groups, deploy the latest application code, and restore any in-memory caches or session state, which can easily exceed the 2-hour RTO. The cross-Region database replication satisfies the 15-minute RPO, but the unproven scaling and initialization steps make pilot light unreliable for this requirement unless you have pre-warmed, fully automated runbooks that can complete within two hours.

Backup and restore only: rely on daily automated backups and restore into the secondary Region during an incident.

Why it fails: Backup and restore relies on periodic snapshots (e.g., daily RDS automated backups) being copied to the secondary Region and restored only after a disaster is declared. Since backups are captured once a day, you risk losing up to 24 hours of transactions, which violates the 15-minute RPO; restoration also requires creating new database instances and reloading data, a process that often takes longer than 2 hours for production-sized PostgreSQL databases. This is the cheapest DR pattern, but it is only acceptable for non-critical workloads with high RTO and RPO tolerance, not for financial services.

Multi-site active-active: run both Regions at full capacity and route live traffic to both simultaneously.

Why it fails: Multi-site active-active runs full production capacity in both Regions and splits live user traffic between them simultaneously, typically with bi-directional database replication to make both copies writable. For PostgreSQL, this requires either an application-level replication strategy or a multi-writer configuration, which introduces complexity around conflict resolution and consistency that is hard to manage; even if you use a single writer with cross-Region replication, you lose the true active-active data tier. The cost of running double capacity, plus the need for automated traffic management and database conflict handling, far exceeds the budget and is over-engineering when a simpler warm standby can meet the required RPO and RTO.

Turn on S3 Transfer Acceleration for the bucket.

Why it fails: S3 Transfer Acceleration uses AWS edge locations and optimized network paths to speed up large uploads into a bucket from distant clients, but it does not alter how data is stored or protected. It only affects the data transfer path, not the durability or availability of the objects once they reside in S3. Enabling it has no effect on accidental deletion, overwrites, or regional failures, so it is irrelevant to recovery of the financial exports.

Use only lifecycle policies to move objects to Glacier.

Why it fails: Lifecycle policies let you define rules to transition objects to colder storage classes like S3 Glacier or expire them after a specified age, which primarily helps manage storage costs. They do not create additional copies of data, do not retain previous versions, and do not protect against accidental deletion — in fact, a lifecycle expiration rule could even permanently remove objects if misconfigured. Using only lifecycle policies focuses on cost optimization, not on data protection or disaster recovery, so it cannot satisfy the requirement to recover from destructive actions or regional outages.

Enable S3 Block Public Access on the bucket.

Why it fails: S3 Block Public Access is a set of security controls that prevent public exposure of objects by overriding bucket policies and ACLs. It protects data from unauthorized internet access, but it does not safeguard against accidental deletion by internal users or compromised credentials. It also does not create copies in another region, so even if data remains private, it could still be lost permanently due to a delete or a regional outage. Thus it is a security measure, not a data protection or recovery mechanism.

Keep using the original writer instance endpoint so the database host name never changes during failover.

Why it fails: Pinning the application to the original writer instance endpoint is fragile because that endpoint is tied to a specific instance, not the writer role. When a failover promotes a different Aurora replica to become the new writer, the old instance is demoted to a reader, so the original endpoint now points to a node that rejects write statements. The database host name effectively changes on failover, and the application would need to discover the new writer instance, defeating the purpose of Aurora's managed endpoints.

Convert the Aurora cluster to Single-AZ so there is only one database node to connect to.

Why it fails: Converting to a Single-AZ Aurora deployment removes the standby replicas that enable automatic failover, leaving only one database node in a single Availability Zone. A storage or compute failure in that AZ would cause a full outage with no standby to promote, and maintenance events would also interrupt availability. This approach reduces resilience and does nothing to address how the application connects during failover; it actually makes connection interruption more likely.

Place Route 53 in front of the database and manually update DNS records whenever failover occurs.

Why it fails: Manually updating Route 53 records on every failover is slow and error-prone, because Aurora already manages the cluster and reader endpoints, which automatically change their DNS resolution to point to the correct instances after promotion. The manual process would extend recovery time and requires external automation or human intervention. In contrast, using the built-in Aurora endpoints eliminates the need for any DNS-level customization, so Route 53 adds no benefit for this use case.

Backup and restore

Why it fails: Backup and restore incurs the lowest cost by storing snapshots and artifacts in S3, but during a regional failure you must provision entirely new infrastructure (e.g., via CloudFormation) and restore data from RDS snapshots or EBS volumes before traffic can be served. This manual rehydration and provisioning process typically pushes RTO well beyond 2 hours, especially for large databases or complex application stacks, so it fails the stated recovery objective.

Multi-site active-active

Why it fails: Multi-site active-active operates production workloads in two or more Regions simultaneously, distributing live traffic to all sites and enabling immediate failover with near-zero RTO. This delivers excellent availability but requires redundant infrastructure, data synchronization, and complex traffic management, resulting in significantly higher costs and operational overhead than needed for a 2-hour recovery window. It overshoots the stated requirement, so it is not the best selection.

Single-AZ deployment

Why it fails: A single-AZ deployment places all resources in one Availability Zone, which is an availability design within a single Region and cannot survive an entire regional outage. Although it provides some protection against AZ-level failures, it offers no cross-Region redundancy or failover capability, so it fundamentally fails to meet the requirement to recover from a regional disaster. It is not considered a disaster recovery strategy at all.

Place the Auto Scaling group in only one subnet so instance launches are simpler.

Why it fails: A single subnet confines the Auto Scaling group to one Availability Zone, so that zone's failure eliminates every instance and blocks replacement launches. It is tempting because one subnet simplifies configuration, and it would be correct for non-production workloads where cross-zone resilience is not required.

Increase the target group deregistration delay to allow old instances to stay longer.

Why it fails: Deregistration delay only controls how long the ALB waits before removing a draining target; it does nothing to launch replacement capacity. It is tempting because it smooths rolling deployments and connection draining, but the stem requires the Auto Scaling group to span multiple Availability Zones so instances launch automatically when one zone fails.

Use a Network Load Balancer, but keep all subnets in a single Availability Zone.

Why it fails: A Network Load Balancer still requires healthy targets across Availability Zones; confining every subnet to one zone leaves no capacity elsewhere, so an AZ outage removes all targets. It is tempting because NLB handles Layer 4 traffic, and it would be correct for TCP or ultra-low-latency workloads spanning multiple zones.

Use weighted routing with a 50/50 traffic split between both Regions.

Why it fails: Weighted routing splits traffic by assigned proportions and does not automatically divert all new clients to the secondary when the primary endpoint fails. It is tempting as a simple distribution mechanism, and would be correct for gradual canary or blue/green traffic shifting, not health-driven failover.

Use latency-based routing so clients always choose the fastest Region.

Why it fails: Latency-based routing selects the lowest-latency Region per client and does not implement an active-passive primary preference with health-based failover. It is tempting because it improves performance across Regions, and would be correct when the goal is latency optimisation rather than primary/secondary failover.

Use a geolocation policy without health checks.

Why it fails: Geolocation routing maps clients by geographic origin and, without health checks, cannot detect an unhealthy primary endpoint or redirect traffic. It is tempting for data-residency or localisation requirements, and would be correct when users must reach a Region based on their location.

Backup and restore only, with no continuously running infrastructure in the secondary Region.

Why it fails: Backup and restore cannot meet a one-hour RPO, since recovery points are periodic snapshots and rebuilding infrastructure typically exceeds six hours. It tempts because it is the cheapest option for workloads tolerating multi-hour RTO and RPO, where no standby capacity is justified.

Pilot light, keeping only the minimum resources needed to bootstrap the environment.

Why it fails: Pilot light keeps only minimal bootstrap resources running, so recovering to full serving capacity typically exceeds the six-hour RTO. It tempts because it suits scenarios needing low-cost standby with longer RTO windows, where core data services replicate but compute stays dormant.

Multi-site active-active, serving production traffic from both Regions at all times.

Why it fails: Active-active runs full production capacity in both Regions continuously, contradicting the requirement to avoid that cost. It tempts because it delivers near-zero RTO and RPO, making it right when the business demands uninterrupted serving and accepts duplicated infrastructure spend.

Invoke Lambda directly from EventBridge in asynchronous mode, and increase the Lambda timeout to reduce failures.

Why it fails: Invoking Lambda directly from EventBridge asynchronously does provide built-in retries, but those retries are limited to the function's retry behavior and do not include a queue-based buffer; if the Lambda is throttled or crashes repeatedly, events are dropped after the retry budget is exhausted. Increasing the timeout only reduces failures caused by functions running out of time, not failures from downstream database timeouts, non-idempotent duplicate processing, or application bugs, so it does not give a place to retain and replay failed events. Without a DLQ or a separate pipeline, there is no systematic way to recover lost messages, making this unreliable for order processing.

Use SNS topics with Lambda subscriptions, but remove all retry and DLQ configuration to minimize duplicate events.

Why it fails: SNS with a Lambda subscription can provide asynchronous fan-out, but deliberately removing all retry and DLQ configuration means any event that causes the Lambda to throw an exception is immediately discarded, since SNS will not redeliver after retries are disabled. The absence of a DLQ leaves no audit trail of unprocessable events, so a transient failure in the database or a bug in the code silently loses the order. SNS also does not offer queue-based buffering or receive-count semantics like SQS, so this design provides neither delivery guarantees nor a recovery mechanism and is inferior to an SQS-based pipeline for critical order processing.

Store failed events only in CloudWatch logs, and have operators manually copy log entries back into the database for reprocessing.

Why it fails: Storing failed events only in CloudWatch logs conflates a monitoring and observability system with a durable message store; logs are append-only, often distributed, and not structured for programmatic retrieval, so extracting events for replay requires parsing unstructured output. Manually copying log entries back into a database is error-prone, slow, and creates a risk of duplication or omission, especially under a high volume of failures during an incident. This approach lacks automation, transactional consistency, and a defined replay workflow, making it operationally untenable for production order processing.

Use a single Route 53 A record that points only to Region A’s ALB and manually update it after failures.

Why it fails: A single-record setup does not provide automatic failover and still requires manual changes during outages.

Use Route 53 latency-based routing with separate records for each region.

Why it fails: Latency-based routing optimizes user proximity, but it does not automatically fail over based on health checks.

Use weighted routing and set the Region B weight to 0 to ensure it is only used when needed.

Why it fails: Weighted routing does not react to health status. Weight changes are not automatic failover responses.

Send events directly from EventBridge to Lambda without any queue to simplify the flow.

Why it fails: EventBridge-to-Lambda direct invocation offers no buffer, so database throttling still surfaces as Lambda retries and eventual message loss, with no isolation of repeatedly failing events. It is tempting because EventBridge rules are a standard event-routing pattern, but the scenario explicitly requires buffering plus a quarantine mechanism, which an SQS queue with a dead-letter queue provides.

Use SNS fan-out to multiple Lambda functions, but keep no retry logic and no DLQ.

Why it fails: SNS fan-out delivers each message to every subscriber and, without a DLQ or retry policy, discards messages that Lambda cannot process, so throttled orders are still lost. SNS suits parallel notification to many independent consumers, not buffering a single ordered event stream with failure isolation.

Store events in an S3 bucket and trigger Lambda immediately after each upload, without using DLQs.

Why it fails: S3-triggered Lambda still invokes synchronously with no retry buffer, and omitting a dead-letter queue means repeatedly failing orders are discarded rather than isolated for inspection. It is tempting because S3 event notifications are a familiar serverless trigger, but the requirement is buffering with automatic retries and quarantine, which SQS with a DLQ satisfies.

Use an RDS single-AZ instance and periodically restore from automated backups if needed.

Why it fails: A single-AZ RDS deployment has no standby instance, so if the underlying AZ becomes unavailable the database endpoint remains down until an operator intervenes. Restoring from automated backups is a disaster recovery procedure, not high availability: you must create a new instance and repoint the application, and point-in-time recovery can lose transactions committed after the last restore time. This produces significant RTO and measurable RPO, far worse than the near-zero RTO of automatic Multi-AZ failover.

Create a read replica in a different AZ and use it only when the primary fails.

Why it fails: A read replica is an asynchronous copy of the primary intended to offload SELECT traffic, not to act as a failover target; because replication lag exists, promoting it can lose recent writes. To use it during an outage you must manually call promote-read-replica, wait for promotion to complete, and reconfigure the application to point to the new endpoint. It also does not automatically monitor or replace the failed primary, so it cannot deliver the same automatic recovery behavior as a Multi-AZ standby.

Use RDS with Multi-AZ disabled, but increase storage IOPS to prevent failover.

Why it fails: Increasing storage IOPS only improves throughput and latency characteristics; it has no effect on Availability Zone availability and does not create a second database for failover. Multi-AZ-disabled RDS runs a single primary instance in one AZ, so if that AZ is lost the database is unreachable regardless of how fast the EBS volume performs. I/O capacity cannot replace the physical redundancy required for automatic failover, making this option orthogonal to the stated availability requirement.