Resolving Memory Bottlenecks on SageMaker Endpoints
A company operates a real-time fraud detection system using SageMaker. The model is deployed on an ml.c5.xlarge instance behind an Application Load Balancer (ALB). Recently, during a sales event, traffic spiked and the endpoint returned HTTP 503 errors. The team scaled the instance count from 2 to 5, but errors persisted. CloudWatch metrics show low CPU utilization (~30%) and high memory usage (~90%). The model loads a large dictionary file (2GB) into memory at startup. Which action should resolve the issue?
Quick Answer
The CloudWatch signature in this scenario, low CPU utilization around 30% paired with high memory usage around 90%, is the key diagnostic clue, and it rules out anything that only adds compute or instance count without addressing memory headroom. The model loads a 2 GB dictionary into memory at startup, and on an ml.c5.xlarge instance with only 8 GB of total memory, that dictionary plus normal request-handling overhead is enough to push memory usage to the point where the instance can't keep up, producing 503 errors regardless of how many of these memory-constrained instances you add. That's exactly why scaling out from 2 to 5 instances didn't help: each individual instance is still memory-bound, so multiplying identical, undersized instances just multiplies the same bottleneck rather than removing it. Moving to a c5.2xlarge doubles available memory to 16 GB while also adding more CPU cores, giving the instance enough headroom to hold the dictionary and serve concurrent requests without hitting the memory ceiling again. The other options either don't address memory at all (Spot instances, more c5.xlarge instances) or trade away compute capacity to get memory (a smaller-vCPU, memory-optimized type), which risks creating a new CPU bottleneck. Whenever CloudWatch shows low CPU alongside high memory utilization on a compute-optimized instance, treat that as a signal to look for a larger or memory-optimized instance rather than simply adding more of the same undersized instance.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Switch to a compute-optimized instance type like c5.2xlarge.
The high memory usage (~90%) with low CPU (~30%) indicates the instance is memory-constrained under load. The ml.c5.xlarge has 8 GB memory, and the model's 2 GB dictionary plus overhead exhausts memory, causing 503 errors. Scaling out doesn't help because each instance is individually memory-bound. Option B switches to c5.2xlarge, which provides 16 GB memory (doubling capacity) and more CPU cores, addressing both memory exhaustion and ensuring sufficient compute for concurrent requests. Option A (Spot instances) does not increase per-instance memory. Option C (scaling to 10 instances) still uses c5.xlarge instances with 8 GB each, so each remains memory-limited. Option D (r5.large) offers 16 GB memory but reduces vCPUs to 2, which could create a CPU bottleneck for the inference workload, making B the better choice.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Enable auto-scaling with Spot instances.
Why it's wrong here
Auto-scaling with Spot instances does not increase the memory per instance and may introduce interruptions, so it does not resolve the memory bottleneck.
- ✓
Switch to a compute-optimized instance type like c5.2xlarge.
Why this is correct
Switching to c5.2xlarge doubles memory (16 GB) and increases CPU cores, directly addressing the memory exhaustion and providing headroom for concurrent requests.
- ✗
Increase the number of instances further to 10.
Why it's wrong here
Increasing the instance count to 10 still uses ml.c5.xlarge instances with 8 GB each, so each instance remains memory-constrained, and errors will persist.
- ✗
Use a memory-optimized instance type like r5.large.
Why it's wrong here
While r5.large has 16 GB memory, it has only 2 vCPUs, which may bottleneck inference throughput; a compute-optimized instance with proportionate memory (c5.2xlarge) is more balanced for this workload.
Visual reference
Go deeper
Related to this question
About these practice questions
This MLS-C01 question is part of Courseiva's 1,672-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on MLS-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A company uses SageMaker to host a real-time inference endpoint. The endpoint is receiving a large number of requests, but the latency is higher than expected. The data scientist observes that the CPU utilization is low but memory utilization is high. Which action should be taken to reduce latency?
easy- ✓ A.Switch to an instance type with more memory or optimize the model to reduce memory footprint.
- B.Enable VPC traffic mirroring to diagnose network issues.
- C.Use an instance type with more vCPUs.
- D.Increase the number of instances in the endpoint.
Why A: High memory utilization indicates the model is memory-bound. Increasing instance memory or optimizing the model to reduce memory footprint can reduce latency. Option B is wrong because VPC traffic mirroring is used for network diagnostics, not for addressing memory bottlenecks. Option C is wrong because CPU utilization is low, so adding more vCPUs would not help; the bottleneck is memory, not CPU. Option D is wrong because increasing the number of instances can improve throughput but does not directly reduce per-request latency for a memory-bound model; it may also increase cost.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.