MLA-C01 Deployment and Orchestration of ML Workflows Practice Question
A company is using SageMaker to serve a model for real-time predictions. They want to test a new model version by routing a small percentage of live traffic to it while the rest goes to the current model. They also need to compare performance metrics. Which TWO actions should they take? (Select TWO.)
⚠ Common exam trap
A common misconception is that you need an external load balancer or DNS service (like Route 53) to split traffic between model versions, but SageMaker's built-in production variant feature handles this natively.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Monitor the performance of both variants using SageMaker CloudWatch metrics
Amazon CloudWatch provides built-in metrics for SageMaker endpoints, including latency, invocation counts, and error rates, which can be monitored per production variant. This allows the company to compare the performance of the new model version against the current model in real time. Option E is correct because SageMaker endpoints support multiple production variants, and you can set an initial traffic weight (e.g., 5%) to route a small percentage of live traffic to the new model while the rest goes to the existing variant.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Deploy the new model to a separate endpoint and use Route 53 to split traffic
Why it's wrong here
Route 53 splits DNS traffic at the domain level, so requests reach one endpoint or the other without per-variant invocation metrics or weighted model-level routing. It is tempting for blue/green or latency-based failover across regions, and would be correct for geographic or availability routing, not model A/B testing.
- ✗
Compile the new model with SageMaker Neo before deployment
Why it's wrong here
Neo compiles a model for optimised inference on target hardware; it does not split live traffic or collect comparative metrics. It is tempting because compilation reduces latency and model size, and would be right when deploying a single model to edge or cost-sensitive endpoints, not for A/B routing.
- ✗
Use SageMaker Batch Transform to evaluate the new model
Why it's wrong here
Batch Transform scores an entire dataset offline in one job, so it cannot route a percentage of live real-time requests or gather concurrent production metrics. It is tempting for bulk offline evaluation, and would be correct when validating a model against a stored dataset rather than live traffic.
- ✓
Monitor the performance of both variants using SageMaker CloudWatch metrics
Why this is correct
SageMaker publishes per-variant invocation, latency and error metrics to CloudWatch, enabling direct comparison of the new model against the current one. This satisfies the requirement to compare performance metrics across both variants during the live traffic test.
- ✓
Configure a production variant with the new model and set initial traffic weight to a small percentage
Why this is correct
Adding the new model as a production variant with a small initial traffic weight implements SageMaker's built-in traffic splitting, routing a defined percentage of live requests to the new version while the existing variant serves the remainder.
Go deeper
Related to this question
About these practice questions
Courseiva writes every MLA-C01 question from scratch — 665 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLA-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLA-C01 exam.