MLS-C01 Practice Question: Machine Learning Implementation and Operations
An ML team is deploying a model for real-time inference. They require A/B testing to compare a new model against the existing one. Which THREE steps should they take to set up this test?
⚠ Common exam trap
Candidates often confuse batch inference (Batch Transform) with real-time inference (endpoint variants), or mistakenly think CloudWatch alarms can directly control traffic routing, when in fact traffic weights are set manually or via SDK/CLI updates.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Set up a second production variant for the existing model
To perform A/B testing with SageMaker, you must create a second production variant for the existing model so that both the old and new models are deployed as separate variants under the same endpoint. This allows the endpoint to serve both models simultaneously and compare their performance.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Set up a second production variant for the existing model
Why this is correct
Both models must be variants to split traffic.
- ✗
Set up a SageMaker Batch Transform job for each model
Why it's wrong here
Batch Transform is for offline predictions.
- ✗
Configure CloudWatch alarms to trigger variant switching
Why it's wrong here
Alarms are for monitoring, not traffic routing.
- ✓
Configure the endpoint to route a percentage of traffic to each variant
Why this is correct
Traffic splitting enables A/B testing.
- ✓
Create a SageMaker production variant for the new model
Why this is correct
A production variant represents a model in the endpoint.
Go deeper
Related to this question
About these practice questions
One of 1,672 original MLS-C01 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.