MLA-C01 Deployment and Orchestration of ML Workflows Practice Question
A team uses SageMaker real-time endpoints for inference. They want to deploy a new model version and compare its performance with the current version under live traffic without affecting user experience. Which method should they use?
⚠ Common exam trap
MLA-C01 often tests the difference between shadow testing (no user impact, mirrored traffic) and A/B testing (real user impact, split traffic) — candidates pick A/B testing because both compare models under live traffic, but only shadow testing guarantees zero user exposure.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Shadow testing with SageMaker
Shadow testing with SageMaker deploys the new model alongside the production model and mirrors live traffic to it without returning its responses to users. This lets the team compare performance under real traffic with zero impact on user experience.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
A/B testing with production variant traffic splitting
Why it's wrong here
A/B testing splits live traffic between variants, so some users receive the unvalidated model, contradicting the requirement to avoid affecting user experience. It is tempting because it compares versions, but shadow testing mirrors traffic to the new model without serving its predictions.
- ✗
Batch transform on a holdout test set
Why it's wrong here
Batch transform scores a static holdout dataset offline, so it never receives live production traffic and cannot compare versions under real conditions. It is tempting because it validates a model cheaply before deployment, which suits offline evaluation rather than live A/B measurement.
- ✗
Blue/green deployment
Why it's wrong here
Blue/green shifts all traffic to the new version after validation, so it cannot run both versions concurrently under live traffic for comparison. It is tempting because it isolates the new model from users, which suits a full cutover with instant rollback rather than side-by-side measurement.
- ✓
Shadow testing with SageMaker
Why this is correct
Shadow testing deploys the new model variant alongside the production variant, mirroring live traffic to it without returning its responses to users. This satisfies the stem's requirement to compare performance under live traffic while leaving user experience unaffected.
Go deeper
Related to this question
About these practice questions
This MLA-C01 question is part of Courseiva's 665-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official Amazon Web Services exam blueprint
This MLA-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLA-C01 exam.