PMLE Serving and Scaling Models Practice Question
Your team has deployed a model on Vertex AI endpoints and you are planning an A/B test to compare a new challenger model (v2) against the current champion (v1). The test should measure business metrics such as click-through rate. Which THREE steps should you take to set up the A/B test correctly? (Choose 3 correct answers)
⚠ Common exam trap
A common mix-up: candidates confuse Vertex AI Experiments (for training) with endpoint traffic splitting (for serving), and they incorrectly think creating separate endpoints with DNS shifting is a valid A/B testing method, when Vertex AI's native traffic splitting is the correct and simpler approach.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Deploy the challenger model (v2) to the same endpoint as the champion (v1).
Option A is correct because Vertex AI endpoints natively support deploying multiple models to the same endpoint, which is the prerequisite for serving both champion v1 and challenger v2 behind a single prediction URL. Option E is correct because once both models share an endpoint, you configure a traffic split (e.g., 90% to v1 and 10% to v2) via the deployed model's traffic percentage, which is exactly how Vertex AI routes online prediction requests between model versions for an A/B test. Option B is correct because to measure business metrics such as click-through rate per variant, the application must log which model version (or deployed model ID) served each prediction so that downstream clicks can be attributed to v1 or v2. Option C is not appropriate because shifting DNS traffic is a coarse, infrastructure-level approach that does not use Vertex AI's built-in traffic splitting and cannot reliably target a percentage of prediction requests. Option D is not appropriate because Vertex AI Experiments is designed for tracking and comparing training/evaluation runs and metrics, not for routing live prediction traffic between deployed model versions.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Deploy the challenger model (v2) to the same endpoint as the champion (v1).
Why this is correct
Deploying both models to the same endpoint is required for A/B testing, because Vertex AI traffic splitting operates across model versions co-deployed on one endpoint. This lets requests be routed proportionally to v1 and v2 for click-through-rate comparison.
- ✓
Modify your application to log which model version served each prediction.
Why this is correct
Logging which model version served each prediction is essential: it links predictions to v1 or v2 so click-through rate can be attributed per variant. Without this label, the business metric cannot be compared between champion and challenger.
- ✗
Create a new endpoint for v2 and gradually shift DNS traffic.
Why it's wrong here
Shifting DNS traffic to a new endpoint bypasses Vertex AI’s built-in traffic-splitting mechanism, which is required to serve both v1 and v2 from the same endpoint and log per-model metrics for a controlled A/B test. This approach is tempting because it mirrors a standard blue/green deployment pattern for stateless web services, where gradual DNS cutover is a valid strategy for zero-downtime releases. However, it fails here because Vertex AI endpoints natively support percentage-based traffic routing between model versions, enabling direct comparison of business metrics like click-through rate without altering DNS infrastructure.
- ✗
Use Vertex AI Experiments to compare model performance.
Why it's wrong here
Vertex AI Experiments tracks and compares training runs and metrics, not live traffic splits or business outcomes such as click-through rate. It is tempting because it is the Vertex AI comparison tool, and it would be correct for evaluating model versions offline before deploying them to an endpoint for online A/B testing.
- ✓
Set up a traffic split between v1 and v2, e.g., 90% v1 and 10% v2.
Why this is correct
A traffic split such as 90% v1 and 10% v2 routes live requests to both deployed models on the endpoint, exposing the challenger to real users. This produces the click-through data needed to compare champion and challenger.
Go deeper
Related to this question
About these practice questions
Courseiva writes every PMLE question from scratch — 775 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This PMLE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PMLE exam.