PMLE Serving and Scaling Models Practice Test 8 — 15 Questions

Question 1

Your team has deployed a model on Vertex AI endpoints. You need to monitor the prediction latency to ensure it meets a 99th percentile SLO of 500ms. You want to set up an alert if the latency exceeds this threshold. Which metric should you use?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The 99th percentile of the `prediction/online/response_latencies` metric.. The `prediction/online/response_latencies` metric in Vertex AI provides a distribution of latency values, allowing you to query the 99th percentile directly. This aligns with the SLO requirement to monitor the tail latency, not the average or maximum, ensuring that the worst-case performance for 1% of requests stays under 500ms.

Answer

The number of prediction requests that timeout.

Answer

Average prediction latency from the endpoint's logs.

Answer

The maximum prediction latency from the endpoint's monitoring dashboard.

Question 2

Your company runs a high-traffic web application that serves the same machine learning model prediction for many identical requests (e.g., product recommendations for the same user profile). You want to reduce latency and load on the prediction endpoint by caching responses. Which Google Cloud service should you use?

Accepted Answer

Cloud Memorystore. Cloud Memorystore (B) is correct because it provides a managed in-memory cache (Redis or Memcached) that can store the results of identical prediction requests, reducing latency and load on the prediction endpoint. By caching responses keyed on the user profile or request parameters, subsequent identical requests can be served directly from Memorystore in microseconds, avoiding redundant model inference.

Answer

Cloud CDN

Answer

Cloud Spanner

Answer

BigQuery

Question 3

You have a TensorFlow model that you want to deploy on edge devices for real-time inference. The model was trained in Vertex AI. You need to convert it to a format suitable for on-device inference. Which approach should you use?

Accepted Answer

Convert the model to TensorFlow Lite using the TensorFlow Lite converter.. TensorFlow Lite is specifically designed for on-device inference on edge devices, offering optimized performance and reduced model size. The TensorFlow Lite converter transforms a TensorFlow model (e.g., from a SavedModel) into the FlatBuffer format (.tflite), which is lightweight and compatible with mobile and embedded platforms. This directly addresses the requirement for real-time inference on edge devices.

Answer

Export the model as a serialized TFX pipeline.

Answer

Export the model to a SavedModel and deploy it using Vertex AI Edge Manager.

Answer

Use Vertex AI Model Optimization to compile the model for edge devices.

Question 4

You need to run batch predictions on a large dataset stored in BigQuery using a Vertex AI model. The dataset contains 10 million rows, and each prediction takes about 100ms. You want to minimize cost and execution time. What should you do?

Accepted Answer

Use Vertex AI batch prediction with BigQuery as the source and sink.. Vertex AI batch prediction natively supports BigQuery as both input and output, eliminating the need for data export or custom pipelines. For 10 million rows at 100ms each, batch prediction processes them in parallel across multiple machines, minimizing execution time while avoiding the per-node costs of online prediction or the overhead of managing Dataflow or GKE clusters.

Answer

Export the BigQuery data to CSV in GCS, then run a custom Dataflow pipeline to make predictions.

Answer

Use Vertex AI online prediction and send all rows as separate requests.

Answer

Use a custom container running on Google Kubernetes Engine to perform inference.

Question 5

You have a Vertex AI endpoint with two deployed models: a champion (v1) and a challenger (v2). You set the traffic split to 90% v1 and 10% v2. After a week, you observe that v2 has better business metrics. You want to shift all traffic to v2 gradually over 3 days to avoid any risk. What should you do?

Accepted Answer

Update the traffic split configuration on the endpoint multiple times over the 3 days to gradually increase v2's percentage.. Vertex AI endpoints support live traffic splitting between deployed models, allowing you to gradually shift traffic from v1 to v2 by updating the traffic split configuration multiple times over the 3-day period. This approach minimizes risk by enabling incremental rollouts and immediate rollback if issues arise, without requiring client-side changes or downtime.

Answer

Deploy v2 to a new endpoint and update your clients to use the new endpoint.

Answer

Use Vertex AI Experiments to compare v1 and v2, then redeploy v2 with 100% traffic.

Answer

Delete v1 from the endpoint so that all traffic automatically goes to v2.

Question 6

You are using Vertex AI Matching Engine for similarity search. Your index has 10 million embeddings of 512 dimensions. The query latency requirement is under 10ms for 99th percentile. Which index type should you choose?

Accepted Answer

Approximate Nearest Neighbor (ANN) index using the ScaNN algorithm.. The ScaNN (Scalable Nearest Neighbors) algorithm is specifically designed for high-dimensional, large-scale similarity search with strict latency requirements. With 10 million 512-dimensional embeddings, an ANN index like ScaNN can achieve sub-10ms query latency at the 99th percentile by trading a small amount of recall for dramatic speed improvements, which is exactly what Vertex AI Matching Engine optimizes for.

Answer

Brute-force index with cosine distance.

Answer

A custom distance-based index using Cloud SQL.

Answer

A tree-based index from scikit-learn deployed as a custom container.

Question 7

You are deploying a large deep learning model on Vertex AI endpoints. The model requires GPU acceleration and you want to minimize cold-start latency. Which TWO actions should you take? (Choose 2 correct answers)

Accepted Answer

Use a custom container that loads the model during startup.. Option B is correct because using a custom container that loads the model during startup lets you control and optimize the initialization process (for example, preloading weights and warming up the GPU) so the model is ready as soon as the container starts, which directly reduces cold-start latency. Option E is correct because setting minReplicaCount to 1 keeps at least one replica always provisioned and running on Vertex AI, so incoming requests hit an already-loaded model instead of triggering a new replica spin-up, eliminating the cold start entirely. Option A is wrong because setting minReplicaCount to 0 enables scale-to-zero, meaning replicas are torn down when idle and every new request incurs a full cold start, which is the opposite of the goal. Option C is wrong because increasing maxReplicaCount only raises the ceiling for horizontal scaling under load and does nothing to reduce the latency of starting an individual replica. Option D is wrong because batch prediction is an asynchronous, job-based mode that does not serve real-time online requests and therefore is irrelevant to minimizing endpoint cold-start latency.

Answer

Set minReplicaCount to 0 to allow scale-to-zero.

Answer

Increase maxReplicaCount to a high number.

Answer

Use batch prediction instead of online prediction.

Question 8

Your team has deployed a model on Vertex AI endpoints. You need to monitor the prediction latency to ensure it meets a 99th percentile SLO of 500ms. You want to set up an alert if the latency exceeds this threshold. Which metric should you use?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Deploy the challenger model (v2) to the same endpoint as the champion (v1).. Option A is correct because Vertex AI endpoints natively support deploying multiple models to the same endpoint, which is the prerequisite for serving both champion v1 and challenger v2 behind a single prediction URL. Option E is correct because once both models share an endpoint, you configure a traffic split (e.g., 90% to v1 and 10% to v2) via the deployed model's traffic percentage, which is exactly how Vertex AI routes online prediction requests between model versions for an A/B test. Option B is correct because to measure business metrics such as click-through rate per variant, the application must log which model version (or deployed model ID) served each prediction so that downstream clicks can be attributed to v1 or v2. Option C is not appropriate because shifting DNS traffic is a coarse, infrastructure-level approach that does not use Vertex AI's built-in traffic splitting and cannot reliably target a percentage of prediction requests. Option D is not appropriate because Vertex AI Experiments is designed for tracking and comparing training/evaluation runs and metrics, not for routing live prediction traffic between deployed model versions.

Answer

Create a new endpoint for v2 and gradually shift DNS traffic.

Answer

Use Vertex AI Experiments to compare model performance.

Question 9

A machine learning engineer wants to deploy a trained model to Vertex AI for online predictions. Which Vertex AI resource is required to serve the model and provide an endpoint URL?

Accepted Answer

Vertex AI Endpoint. Vertex AI Endpoint is the required resource to deploy a trained model for online predictions, as it provides a dedicated endpoint URL that accepts prediction requests and routes them to the model. Without an endpoint, the model cannot be accessed via HTTP/HTTPS for real-time inference, which is the core requirement for online serving.

Answer

Vertex AI Pipeline

Answer

Vertex AI Model Registry

Answer

Vertex AI Feature Store

Question 10

A company needs to serve a high-throughput prediction service with strict latency requirements. They want to minimize cold starts and ensure consistent performance. Which endpoint configuration is most appropriate?

Accepted Answer

Set min_replicas to an estimated baseline and max_replicas to a higher number. Setting min_replicas to an estimated baseline ensures that a minimum number of instances are always running, eliminating cold starts for baseline traffic. Setting max_replicas to a higher number allows the service to scale up to handle traffic spikes while maintaining consistent performance. This configuration balances cost and latency by avoiding the overhead of scaling from zero while still accommodating bursts.

Answer

Set min_replicas and max_replicas equal to a fixed number

Answer

Set min_replicas to 0 and max_replicas to a high number

Answer

Do not set min_replicas; let Vertex AI automatically determine

Question 11

A data scientist wants to perform A/B testing between two model versions deployed on the same Vertex AI endpoint. They need to route 10% of traffic to the challenger model. Which approach should they use?

Accepted Answer

Deploy both models to the same endpoint and set traffic_split to 90 for champion and 10 for challenger. Vertex AI endpoints support traffic splitting directly, allowing you to route a percentage of requests to different model versions deployed on the same endpoint. By setting `traffic_split` to 90 for the champion and 10 for the challenger, the data scientist can perform online A/B testing without additional infrastructure. This is the simplest and most cost-effective approach, as it avoids managing separate endpoints or load balancers.

Answer

Use Vertex AI Experiments to compare models offline, then deploy the winner

Answer

Deploy the challenger model to a separate endpoint and use a load balancer to split traffic

Answer

Update the champion model with a new version and use model version aliases

Question 12

A team is deploying a large PyTorch model for online inference. They want to use NVIDIA Triton Inference Server to optimize serving performance. How can they integrate Triton with Vertex AI?

Accepted Answer

Package the model with Triton in a custom container and deploy it to Vertex AI. Vertex AI supports custom containers for prediction, so the standard integration pattern is to build a container that bundles the model with NVIDIA Triton Inference Server and deploy it as a Vertex AI Model with a custom prediction container. This lets Triton handle dynamic batching, model ensembles, and multi-framework serving while Vertex AI manages endpoints, autoscaling, and monitoring.

Answer

Vertex AI automatically uses Triton for all PyTorch models

Answer

Deploy the model to GKE and use Vertex AI as a frontend

Answer

Use a prebuilt Vertex AI PyTorch container that includes Triton

Question 13

A company needs to run batch predictions on 10 TB of data stored in Cloud Storage. The predictions should be written to BigQuery. Which approach should they use?

Accepted Answer

Create a Vertex AI Batch Prediction job with GCS input and BigQuery output. Vertex AI Batch Prediction natively supports reading input from Cloud Storage and writing predictions directly to BigQuery, making it the most efficient and fully managed solution for large-scale batch inference on 10 TB of data. This approach avoids the complexity of custom infrastructure or per-record model calls, leveraging Vertex AI's optimized batch processing pipeline.

Answer

Export the model to Cloud Functions and trigger on file upload

Answer

Use Vertex AI Online Prediction with a batch job

Answer

Use Dataflow to read from GCS and write to BigQuery, calling the model for each record

Question 14

A company uses Vertex AI Matching Engine for a product recommendation system. They need to update the index with new product embeddings every hour, but the index is used for online queries with low latency. Which index update strategy should they use?

Accepted Answer

Use streaming updates to insert new embeddings incrementally. Streaming updates in Vertex AI Matching Engine allow incremental insertion of new embeddings into an existing index without rebuilding it. This satisfies the requirement for hourly updates while maintaining low-latency online queries, as the index remains available and consistent during the update process.

Answer

Use a hybrid approach with batch for daily full rebuild and streaming for hourly

Answer

Use batch updates to replace the index every hour

Answer

Recreate the index from scratch each hour

Question 15

A team has deployed a model on Vertex AI and wants to cache frequent identical prediction requests to improve latency and reduce cost. Which Google Cloud service should they use?

Accepted Answer

Cloud Memorystore. Cloud Memorystore provides a managed Redis or Memcached in-memory cache that can store frequent identical prediction requests and their responses, dramatically reducing latency and backend load. It is the standard Google Cloud service for application-level caching in front of Vertex AI endpoints.

Answer

Cloud Bigtable

Answer

Cloud CDN

Answer

Cloud SQL