Design High-Performing Architectures practice questions

Set the performance mode to General Purpose.

Why it fails: General Purpose mode is the default and is best for latency-sensitive applications like web serving or home directories. However, for 'heavy file operations' that are likely highly parallel, this mode may reach its IOPS limit, making it the cause of the degradation rather than the solution.

Move files to the Infrequent Access (IA) storage class.

Why it fails: The Infrequent Access storage class is designed for cost optimization of data that is not accessed regularly. It has higher latency for first-byte access compared to the Standard storage class, so moving files here would likely decrease performance rather than improve it for an active CMS.

Transition the workload to EBS Multi-Attach volumes.

Why it fails: EBS Multi-Attach allows an io1 or io2 volume to be attached to multiple instances in the same Availability Zone. However, it requires a cluster-aware file system and does not scale across multiple Availability Zones like EFS, making it a complex and restricted alternative for a CMS.

AWS Global Accelerator.

Why it fails: Global Accelerator provides static IP addresses that act as an entry point for global traffic. It uses the AWS global network to route requests to the closest healthy regional endpoint, which is ideal for dynamic applications that cannot leverage traditional caching. This minimizes network latency and packet loss for users.

Amazon Route 53 latency-based routing.

Why it fails: Route 53 latency routing directs users to the region with the lowest latency at the time of DNS resolution. However, once the connection is established, it still relies on public internet routing. Global Accelerator provides a much more stable and lower-latency connection by leveraging the internal AWS global backbone network.

AWS Direct Connect.

Why it fails: Direct Connect is intended for dedicated physical connections between an on-premises data center and AWS. It is not designed to accelerate traffic for global public internet users accessing a web application. Its use case is entirely different from optimizing application performance for a global, internet-based user base.

Add more read replicas to the primary database.

Why it fails: While read replicas help, adding more does not necessarily fix the performance issues caused by heavy analytical queries, which can still lock resources or consume significant CPU. A dedicated architectural shift, such as using an OLAP-optimized engine, is required to prevent analytical workloads from impacting transactional latency.

Migrate both workloads to a larger single instance.

Why it fails: Scaling vertically to a larger instance eventually hits a limit and is extremely expensive. It does not address the fundamental issue that transactional and analytical workloads have different performance profiles. A better architecture involves separating these workloads so each can be optimized according to its specific access pattern.

Implement database sharding on the primary instance.

Why it fails: Sharding is complex and primarily designed to scale write throughput. It does not resolve the issues caused by long-running analytical queries. While sharding can handle scale, the architectural complexity is unnecessary if the primary bottleneck is the mix of analytical and transactional workloads on a single database.

Change the DNS provider to a faster third-party service.

Why it fails: While third-party DNS might be marginally faster, it does not solve the fundamental issue of repeating lookups over the network. The most performant way to handle DNS latency is to eliminate the lookup request entirely by caching the result, rather than just changing which server performs the lookup.

Use Route 53 health checks on all endpoints.

Why it fails: Route 53 health checks are used for availability and fault tolerance, not for improving DNS lookup performance. While necessary for reliability, they do not impact the latency of DNS resolution and would not solve the performance issues described in the scenario.

Migrate to a different VPC configuration.

Why it fails: VPC configuration does not influence the speed of DNS resolution. Changing the VPC settings would not address the overhead caused by frequent DNS lookups. Implementing local caching is the appropriate technical solution to minimize the latency incurred by DNS queries within the application's network traffic.

Increase the number of instances behind the ALB.

Why it fails: Adding more instances handles higher traffic volume but does not solve latency issues caused by the distance between the user and the application region. If a user is far away, they will still experience high network latency regardless of how many instances are behind the load balancer.

Enable ALB cross-zone load balancing.

Why it fails: Cross-zone load balancing ensures traffic is distributed evenly across all instances in all AZs within a single region. It does not address performance issues for users connecting from different global regions. It is a regional configuration meant for balancing internal compute load, not global user connectivity.

Enable HTTP/2 on the Application Load Balancer.

Why it fails: HTTP/2 is a protocol improvement that can help with performance by multiplexing requests over a single connection. However, it does not address the fundamental network latency issue caused by geographic distance. Global Accelerator is the more impactful solution for global performance when compared to protocol-level optimizations.

Provisioned IOPS SSD (io1)

Why it fails: While io1 volumes offer high performance and provisioned IOPS, they are limited to 64,000 IOPS per volume. This capacity is insufficient for the application's requirement of 256,000 IOPS. Using io1 would require complex RAID configurations that increase architectural management overhead and potential points of failure.

General Purpose SSD (gp3)

Why it fails: General Purpose SSD volumes are designed for a wide variety of workloads but max out at 16,000 IOPS per volume. They lack the consistent sub-millisecond latency and high-scale performance needed for high-frequency trading platforms, making them unsuitable for this specific high-performance computing requirement.

Throughput Optimized HDD (st1)

Why it fails: Throughput Optimized HDD volumes are backed by magnetic drives and are designed for sequential workloads like data warehousing and log processing. They offer low IOPS and high latency compared to SSDs, failing to meet the sub-millisecond response times required for high-frequency trading applications.

Target tracking policies do not support CPU-based metrics.

Why it fails: This statement is incorrect because 'ASGAverageCPUUtilization' is one of the standard predefined metrics supported by Auto Scaling. Target tracking is specifically designed to use metrics like CPU utilization or request count to adjust the fleet size automatically.

The policy lacks a cooldown period to prevent rapid scaling.

Why it fails: Target tracking policies include a default cooldown period (warm-up time) to ensure that the ASG does not over-scale while new instances are still coming online. The lack of a visible cooldown in the snippet does not mean it is missing or causing the latency.

The policy should use 'ASGAverageNetworkIn' instead of CPU.

Why it fails: Network input is another valid metric, but it isn't inherently better than CPU for all applications. If the application is CPU-bound, switching to a network metric would likely result in less accurate scaling and even worse performance during high-load periods.

Change the performance mode from General Purpose to Max I/O.

Why it fails: Max I/O mode is optimized for workloads where hundreds or thousands of EC2 instances are accessing the file system. It scales to higher levels of aggregate throughput and operations per second than General Purpose mode, making it the correct choice for large-scale parallel processing.

Switch the throughput mode from Elastic to Bursting.

Why it fails: Bursting throughput scales based on the size of the file system, which may not be sufficient for high-throughput needs if the dataset is small. Elastic throughput is generally better for unpredictable workloads, but the underlying performance mode (Max I/O) is what governs the total capacity limits.

Move the data to an EFS One Zone storage class.

Why it fails: EFS One Zone is a cost-optimization feature that stores data in a single Availability Zone. While it reduces costs, it does not provide higher performance or throughput compared to the EFS Standard storage class; its primary impact is on availability and durability.

Increase the size of the EFS file system by adding dummy data.

Why it fails: In 'Bursting' throughput mode, larger file systems earn more 'burst credits.' However, this is an inefficient 'hack.' Modern EFS features like 'Provisioned Throughput' or 'Elastic Throughput' allow for high performance without artificially inflating the storage size, and Max I/O handles the parallelization.

Enable Enhanced Fan-out for the consumers.

Why it fails: Enhanced Fan-out improves the 'read' throughput for consumers by providing a dedicated 2 MB/s pipe per consumer. However, it does not resolve 'ProvisionedThroughputExceededException' on the 'write' side (ingestion), which is caused by the aggregate limit of the shards during data producers' operations.

Increase the retention period of the stream.

Why it fails: Increasing the retention period allows records to stay in the stream longer before being deleted, which is useful for replaying data. It has no impact on the ingestion or consumption throughput and will not resolve errors related to exceeding the provisioned capacity limits.

Perform a resharding operation to increase the number of shards.

Why it fails: Resharding increases the total capacity of the stream. Since each shard has a 1 MB/s ingress limit, increasing from 10 to 20 shards would double the total ingestion capacity to 20 MB/s, effectively resolving the 'ProvisionedThroughputExceededException' caused by high volume from producers.

Convert the EBS volumes to Provisioned IOPS io2.

Why it fails: While io2 offers high performance, it is significantly more expensive than gp3. Using io2 for unpredictable spikes would require over-provisioning IOPS, leading to unnecessary costs. gp3 is designed specifically to provide cost-effective, scalable performance that can be tuned to meet specific workload requirements without high overhead.

Increase the size of the existing gp2 volumes.

Why it fails: In gp2 volumes, performance scales linearly with size. To gain more IOPS, you would have to increase capacity beyond what is needed for storage, leading to wasted space and costs. gp3 is superior here because it decouples storage capacity from performance metrics, allowing for more granular, cost-effective adjustments.

Use Amazon EFS with Provisioned Throughput mode.

Why it fails: EFS is a shared file system and has different performance characteristics compared to block storage. If the application requires EBS-style block storage performance, EFS may introduce higher latency and isn't the direct solution to EBS-specific I/O wait times. Moving to gp3 provides the necessary block-level performance improvements more effectively.

Enable Performance Insights.

Why it fails: Performance Insights helps identify which queries are causing high CPU, but it does not fix the performance issue itself. While useful for analysis, it is a monitoring tool, not a solution to the immediate resource exhaustion indicated by the 98% CPU utilization and low memory availability.

Add a Multi-AZ standby instance.

Why it fails: Multi-AZ is for high availability and failover, not for scaling performance. The secondary instance is not used to process read or write queries during normal operation. Therefore, adding a standby will not reduce the CPU load or alleviate the memory constraints of the primary instance currently experiencing performance degradation.

Decrease the number of Read Replicas.

Why it fails: Read replicas help by offloading reads, but they don't solve the CPU bottleneck on the primary instance. Furthermore, decreasing the number of replicas would likely increase the load on the primary instance by forcing all read requests back to it, worsening the performance issue instead of solving it.

Use an Amazon API Gateway.

Why it fails: API Gateway provides features like request caching, traffic management, and protocol conversion that significantly reduce the complexity and latency of inter-service communication. By acting as a central entry point, it can handle orchestration and caching, reducing the overhead of direct service-to-service calls which often cause performance bottlenecks in microservices.

Increase the number of instances for every service.

Why it fails: Scaling out every service without addressing the underlying communication overhead will likely lead to resource wastage without solving the latency problem. The issue is architectural communication, not necessarily a lack of compute resources. Adding more instances doesn't fix the inefficiencies inherent in the service-to-service interaction patterns.

Switch to a synchronous REST API structure.

Why it fails: Synchronous REST calls are prone to latency issues in microservices due to the chain of dependencies. If one service is slow, it affects all calling services. Moving toward an asynchronous model with an event bus or using an API Gateway for orchestration is generally better for microservice performance and scalability.

Move all services back to a single instance.

Why it fails: Consolidating services back into a monolith negates the benefits of microservices, such as independent scaling and deployment. The goal is to improve the communication performance within the distributed system, not to regress into a monolithic architecture, which creates its own set of performance and scalability limitations at scale.

Amazon CloudFront

Why it fails: Amazon CloudFront is primarily a Content Delivery Network (CDN) designed for caching and delivering HTTP/HTTPS content. While it improves latency for web-based applications, it does not support custom UDP-based protocols at the edge. For gaming applications using UDP, Global Accelerator is the superior choice for optimizing network performance and routing at the transport layer.

AWS Transit Gateway

Why it fails: AWS Transit Gateway is a network hub used to interconnect VPCs and on-premises networks at scale. While it simplifies network management, it does not inherently provide global acceleration for end-user traffic across the public internet. It is more of an internal routing solution than a performance optimization tool for global client-to-server communications.

Application Load Balancer (ALB) with cross-zone load balancing

Why it fails: Application Load Balancers can be used in the backend to distribute traffic across multiple targets, but in the context of UDP, a Network Load Balancer (NLB) is required. However, when paired with Global Accelerator, the NLB ensures that traffic is efficiently handled at the regional level. This combination provides both global path optimization and regional high availability.

Enable Enhanced Networking with the Elastic Network Adapter (ENA).

Why it fails: Enhanced Networking with ENA provides higher bandwidth and lower CPU utilization compared to traditional network interfaces. While it improves performance for general workloads, it does not provide the specialized OS-bypass capabilities required for the most demanding tightly coupled HPC applications found in research environments.

Use a Spread Placement Group for the EC2 instances.

Why it fails: Spread placement groups place each instance on distinct hardware racks to reduce correlated failures. This strategy is excellent for high availability but actually increases network latency between nodes, making it the opposite of what is required for high-performance computing clusters needing low-latency communication.

Implement AWS Global Accelerator for the cluster.

Why it fails: AWS Global Accelerator improves availability and performance for internet-facing applications by routing traffic through the AWS global network. It is not designed for internal inter-node communication within a VPC and would not provide any performance benefits for tightly coupled HPC workloads running on EC2 clusters.

Use a Simple Scaling Policy with a large cooldown period.

Why it fails: Simple scaling policies are less responsive than target tracking because they wait for a specific alarm and cooldown period before taking further action. Large cooldown periods can lead to performance degradation during rapid traffic spikes because the system cannot add new instances quickly enough to meet demand.

Provision IOPS (io2) for all EC2 root volumes.

Why it fails: While provisioned IOPS improve disk performance, they do not address the scaling requirements of a web application facing traffic spikes. Root volumes for web servers typically do not require high IOPS for request handling; instead, the bottleneck is usually CPU, memory, or network bandwidth.

Implement vertical scaling by increasing instance sizes manually.

Why it fails: Vertical scaling requires stopping and starting instances to change their size, which causes downtime and is not a viable strategy for handling unpredictable traffic spikes. Horizontal scaling through Auto Scaling is the preferred AWS architecture for ensuring high performance and availability for web applications.

Enable Multi-AZ deployment for the RDS instance.

Why it fails: Multi-AZ deployment is primarily a high availability and disaster recovery feature that provides a standby instance in a different Availability Zone. It does not provide additional read capacity or improve performance for global users, as all read and write operations still target the primary instance.

Upgrade the RDS instance to a larger instance class.

Why it fails: While upgrading the instance class provides more CPU and memory, it does not solve the fundamental problem of network latency for global users. Even with a massive primary instance, users far from the source region will still experience slow response times due to the physical distance.

Enable Enhanced Monitoring with a 1-second granularity.

Why it fails: Enhanced Monitoring provides detailed metrics about the operating system and processes on the RDS instance. While useful for troubleshooting and identifying performance bottlenecks, it is a diagnostic tool and does not inherently improve the performance or reduce the latency experienced by the end-users.

Increase the DefaultTTL and MinTTL values.

Why it fails: Increasing TTL values determines how long an item stays in the cache once it is already there. However, if every request is treated as unique due to the query string configuration, the items will never be reused regardless of how long their expiration time is set.

Set CookieBehavior to 'all' to include session data.

Why it fails: Including all cookies in the cache key would actually further decrease the cache hit ratio. Since cookies often contain unique session IDs or user-specific data, caching based on them would ensure that almost every request is treated as a unique object, putting even more stress on the origin.

Enable HeaderBehavior for the 'Host' header.

Why it fails: Forwarding headers and including them in the cache key generally reduces the cache hit ratio because it adds another variable that must match for a cache hit. While the Host header is often necessary for multi-site origins, it would not solve the specific performance issue caused by query strings.

Enable Multi-AZ deployment for the existing RDS instance.

Why it fails: Multi-AZ is designed for high availability and disaster recovery, not for performance scaling. While it provides a standby instance, that instance is not accessible for read traffic. Therefore, it does not alleviate the read load on the primary instance or improve read performance for the application.

Increase the instance size to a larger DB instance class.

Why it fails: Scaling vertically by increasing instance size can improve performance, but it is often more expensive than scaling horizontally. Read replicas provide a more cost-effective and scalable approach for read-heavy workloads by allowing you to add multiple read nodes as traffic grows over time.

Migrate the database to Amazon DynamoDB.

Why it fails: Migrating to a NoSQL database like DynamoDB requires a complete rewrite of the application data access layer and schema. This is not a minimal architectural change and is only recommended if the application requires NoSQL-specific features or scale-out capabilities that relational databases cannot provide.

Attach an Internet Gateway to the VPC.

Why it fails: Traffic routed through an Internet Gateway is subject to public network congestion and bandwidth limitations. This approach does not offer the same performance or security characteristics as a VPC Endpoint. Relying on public internet transit is unsuitable for high-throughput applications requiring consistent, high-speed data transfer.

Use a NAT Gateway for all outbound traffic.

Why it fails: NAT Gateways are subject to throughput limits and represent a single point of congestion for traffic headed to S3. They introduce unnecessary latency and potential performance bottlenecks. For high-throughput requirements, private connectivity via an S3 VPC Endpoint is the preferred architectural choice over using NAT services.

Install an AWS Direct Connect connection.

Why it fails: Direct Connect is intended for on-premises to AWS connectivity. While it provides consistent bandwidth, it is not the correct solution for EC2-to-S3 traffic within the same region. Using an S3 VPC Endpoint is the most efficient and cost-effective method for intra-region, high-throughput S3 data access.

Spread Placement Groups.

Why it fails: Spread Placement Groups are designed for high availability by placing instances on distinct physical hardware to minimize the impact of correlated failures. They do not provide the low-latency network performance required by CPU-bound, tightly coupled applications. Instead, they increase the physical distance between instances to improve fault tolerance.

Auto Scaling groups across multiple Availability Zones.

Why it fails: Scaling across multiple Availability Zones increases fault tolerance, but it also introduces network latency between instances residing in different zones. For applications requiring sub-millisecond latency, spreading instances across multiple AZs is counterproductive. These applications generally require a single, high-performance network fabric provided by a Cluster Placement Group.

Dedicated Hosts with physical CPU pinning.

Why it fails: Dedicated Hosts allow for license compliance and physical hardware control, but they do not automatically provide a low-latency network fabric between multiple instances. While they offer performance isolation, the network communication between them still relies on the standard VPC network stack, unlike the optimized performance of Cluster Placement Groups.