PMLE Serving and Scaling Models Practice Test 11 — 15 Questions

Question 1

Your team is serving a large language model on a Vertex AI endpoint using a custom container. You need to reduce inference latency for long prompts while keeping the deployment cost reasonable. The model uses an autoregressive decoder. Which optimization should you implement?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Implement continuous batching and paged attention in the serving container.. For autoregressive LLM serving, the key bottlenecks are GPU underutilization during decoding and memory fragmentation in the key-value cache. Continuous batching keeps the GPU busy by admitting new requests into the current batch, while paged attention stores the KV cache in fixed-size blocks to avoid fragmentation and support more concurrent sequences. These techniques reduce per-token latency and improve throughput, making them the right optimization for long prompts on a cost-conscious deployment.

Answer

Use a smaller quantized version of the model and serve it with a batch size of one.

Answer

Increase maxReplicaCount and rely on autoscaling to handle long prompts.

Answer

Enable response streaming and return tokens as they are generated.

Question 2

A media company is serving a video recommendation model on a Vertex AI Endpoint. The model receives a mix of requests: some require only a few features, while others require many features from a feature store. The team wants to reduce average latency and cost without retraining the model. Which TWO strategies should they use? (Choose two.)

Accepted Answer

Enable request batching on the endpoint to group multiple requests into a single prediction call.. Request batching and feature caching are serving-side optimizations that reduce per-request overhead and repeated feature lookups. They lower average latency and can reduce cost by improving resource utilization, all without retraining the model. Other options either do not address the root causes or are unsuitable for real-time serving.

Answer

Switch from online predictions to batch predictions for all requests.

Answer

Use a smaller machine type for the endpoint to reduce cost per replica.

Answer

Increase the endpoint's maxReplicaCount to handle peak traffic.

Question 3

An ML platform team deploys the same custom container to two Vertex AI endpoints: one for interactive scoring and one for nightly bulk scoring. The interactive endpoint must return predictions in under 200 ms and receives small single-record requests. The bulk endpoint sends large batched requests and tolerates seconds of latency. Both endpoints currently use the same machine type and the same container image, and the interactive endpoint frequently misses its latency target. Which change best resolves the interactive latency problem?

Accepted Answer

Deploy a container variant tuned for the interactive endpoint: load the model once at startup, keep it resident, and avoid per-request batch assembly, then serve it on a latency-optimized machine type.. Interactive single-record scoring and bulk batched scoring have opposite performance profiles, so they should not share one serving configuration. The interactive path is dominated by fixed per-request overhead such as model loading and batch assembly, so a container that loads the model once, keeps it resident, and skips per-request batching removes that cost. Pairing it with a machine type selected for low-latency single requests, while leaving the bulk endpoint tuned for throughput, resolves the SLO breach without over-sizing the bulk tier.

Answer

Increase the request timeout on the interactive endpoint and add more replicas.

Answer

Enable autoscaling on the bulk endpoint so it absorbs the interactive traffic as well.

Answer

Move both endpoints to a machine type with more vCPUs and more memory.

Question 4

You have a custom model deployed on a Vertex AI endpoint that receives online prediction requests. The model expects input features in a specific order, but clients sometimes send features in a different order. You want to ensure that the endpoint consistently receives correctly ordered features without modifying every client. What should you do?

Accepted Answer

Implement a preprocessing function in the model's serving signature that reorders the input features before prediction.. The most effective solution is to incorporate feature reordering into the model's serving logic. By adding a preprocessing function in the serving signature, the endpoint can accept features in any order and rearrange them correctly before inference. This centralizes the logic, avoids client-side changes, and ensures consistent predictions. It is a standard practice in Vertex AI custom containers to include such preprocessing.

Answer

Create a new endpoint for each client that expects the client's specific feature order.

Answer

Configure the endpoint to reject requests with incorrectly ordered features and return an error message to clients.

Answer

Use a Vertex AI Feature Store to serve features to the model, ensuring that features are always in the correct order.

Question 5

A Vertex AI Endpoint hosts a model that must serve predictions with a strict 99th percentile latency under 100 ms. The model is a large TensorFlow model that processes images. During load testing, you observe that p99 latency spikes to 300 ms when batch size exceeds 1. You need to meet the latency SLO while maintaining reasonable throughput. What should you do?

Accepted Answer

Enable dynamic batching with a small maximum batch size and a short timeout.. The latency spike is caused by large batch sizes. Dynamic batching with a small maximum and short timeout limits how long requests wait and how many are combined, keeping p99 under 100 ms while still batching enough to improve throughput. Simply adding replicas or changing hardware does not resolve the batch-size-induced tail latency.

Answer

Increase the batch size further to amortize overhead and improve throughput.

Answer

Increase the number of replicas and keep batch size at 1.

Answer

Switch to a CPU-only machine type with more cores to parallelize inference.

Question 6

A media company uses a Vertex AI endpoint to serve a video recommendation model. The model is updated weekly with new embeddings. They want to minimize downtime during model updates and ensure that the new model performs well before fully rolling it out. They also need to be able to revert quickly if issues arise. What should they do?

Accepted Answer

Deploy the new model as a new version on the same endpoint, set it to receive 0% of traffic initially, then gradually increase traffic while monitoring performance.. The optimal approach is to deploy the new model as a version on the existing endpoint and use traffic splitting to gradually shift traffic. This provides zero-downtime updates, allows performance validation with real traffic, and enables quick rollback by adjusting traffic percentages. Other methods either cause downtime, lack gradual rollout, or add unnecessary complexity.

Answer

Deploy the new model to a new endpoint and use a load balancer to distribute traffic between the old and new endpoints.

Answer

Create a new endpoint for the new model and update the application to call the new endpoint after testing.

Answer

Use a batch prediction job to test the new model on a large dataset, then replace the existing model on the endpoint.

Question 7

A retail team must run nightly batch predictions over 20 TB of Parquet data stored in Cloud Storage using a custom PyTorch model registered in Vertex AI Model Registry. They want the job to finish within a fixed maintenance window and prefer not to manage the underlying compute. Which configuration should they use?

Accepted Answer

Create a Vertex AI BatchPredictionJob with the model from Model Registry, set the input to the Cloud Storage Parquet path, and specify a machine type plus a starting and maximum replica count for the distributed workers.. Batch prediction is purpose-built for large offline scoring: it reads files directly from Cloud Storage, distributes work across managed replicas, and writes sharded output without any endpoint. Specifying machine type and replica bounds gives the team control over throughput so the job completes inside the maintenance window while Vertex AI handles provisioning.

Answer

Schedule the model on a Vertex AI pipeline with a Dataflow step that calls the endpoint for each record, using streaming inserts to write results back to Cloud Storage.

Answer

Deploy the model to a Vertex AI endpoint with autoscaling and write a client script that submits all rows as individual online prediction requests.

Answer

Create a Vertex AI CustomJob that runs a PyTorch training script with the input path as a parameter, since CustomJob automatically performs batch inference when given a Parquet input.

Question 8

A financial services firm serves a fraud-detection model on a Vertex AI endpoint that consumes features from a Vertex AI Feature Store online store. During a load test, prediction latency is acceptable, but the firm discovers that the model's feature values in production drift from the values used at training time because the training pipeline read from a BigQuery table with different transformation logic. The team wants the serving path to use the same feature definitions as training so online and offline values match. Which approach should they take?

Accepted Answer

Define the features once as Feature Store feature views with a shared transformation and serve the model from the online store while the training pipeline reads the same definitions from the offline store.. Training-serving skew from divergent transformation logic is solved by unifying feature definitions, not by tuning storage or adding caching. Vertex AI Feature Store lets a feature view define a transformation once, then materialize the result to the online store for low-latency serving and to the offline store for training. Because both paths derive from the same definition, the values the model sees at inference match the values it was trained on, eliminating the drift at its source.

Answer

Log all online feature values and retrain the model weekly on the logged production data.

Answer

Increase the online store's node count and shorten the feature value TTL so fresher values are served.

Answer

Route serving requests through the offline store and cache results in Memorystore for low latency.

Question 9

You have deployed a model to a Vertex AI Endpoint and need to perform a canary release of a new model version to 10% of traffic. You want to monitor the new version's performance before gradually increasing its traffic share. What should you do?

Accepted Answer

Upload the new model as a new version and deploy it to the same endpoint with 10% traffic split.. Vertex AI Endpoints natively support deploying multiple model versions and splitting traffic by percentage. Deploying the new version to the same endpoint with a 10% split enables canary testing, and the split can be adjusted as confidence grows. Separate endpoints with external load balancing or a full cutover do not meet the controlled, gradual rollout requirement.

Answer

Deploy the new model as a separate endpoint and use a load balancer to split traffic.

Answer

Deploy the new model to the same endpoint with 100% traffic and monitor closely.

Answer

Create a new endpoint for the new model and use Vertex AI's traffic director to shift traffic.

Question 10

A startup is deploying a scikit-learn model to Vertex AI for online predictions. They want to minimize the effort required to containerize the model and ensure it can handle HTTP requests. What should they do?

Accepted Answer

Use the pre-built scikit-learn container provided by Vertex AI and deploy the model by specifying the model artifact in Cloud Storage.. The pre-built scikit-learn container on Vertex AI eliminates the need for custom containerization and HTTP server code. It automatically handles model loading and prediction requests, making it the lowest-effort solution for deploying scikit-learn models.

Answer

Convert the scikit-learn model to TensorFlow SavedModel format and use the TensorFlow pre-built container for deployment.

Answer

Deploy the model using a custom container that includes the scikit-learn library and a simple HTTP server implemented with Python's http.server module.

Answer

Write a custom Flask application to load the model and expose a /predict endpoint, then build a Docker image and push it to Artifact Registry.

Question 11

You have trained a scikit-learn model and want to deploy it to Vertex AI for online predictions. You need to minimize the effort to create a custom container and ensure the model is served with the default pre-built container. What should you do?

Accepted Answer

Save the model using joblib and upload it to Vertex AI Model Registry, then deploy using the pre-built scikit-learn container.. The least-effort path is to use the pre-built scikit-learn container on Vertex AI. Saving the model with joblib and uploading it to Model Registry allows you to deploy without writing any serving code or building a container. The pre-built container handles loading the model and serving predictions. Other options involve unnecessary conversion or custom packaging, which increase effort and risk.

Answer

Package the model into a custom container with a Flask app and deploy it to Vertex AI.

Answer

Export the model as a PMML file and deploy using the pre-built XGBoost container.

Answer

Convert the model to TensorFlow SavedModel format and deploy using the pre-built TensorFlow container.

Question 12

A healthcare analytics team needs to serve a model on Vertex AI to internal applications, but compliance requires that no prediction request or response payload ever be written to logs. They still want basic operational metrics such as request count and latency. What should they configure on the endpoint?

Accepted Answer

Disable request-response logging on the deployed model while leaving endpoint monitoring and Cloud Monitoring metrics enabled.. Payload logging in Vertex AI is controlled by an explicit setting on each deployed model and is off unless enabled. Disabling it prevents request and response content from reaching Cloud Logging, while the endpoint continues to publish aggregate metrics such as request counts, error rates, and latency to Cloud Monitoring, which is exactly the separation this compliance scenario requires.

Answer

Delete the endpoint's service account permissions on Cloud Logging so the platform cannot write any log entries, including metrics.

Answer

Set the endpoint's traffic split to route all requests to a canary deployment that has logging disabled, while the primary deployment keeps logging enabled.

Answer

Enable request-response logging but set the sampling rate to a very small value so almost no payloads are captured.

Question 13

You have a Vertex AI endpoint serving a model that returns predictions in about 200 ms. During a marketing campaign, traffic increases tenfold for short bursts. You want the endpoint to handle the bursts without manual intervention and without over-provisioning for the entire day. What should you do?

Accepted Answer

Configure autoscaling on the endpoint with a minimum replica count of one and a maximum replica count that covers peak load.. Vertex AI endpoints support autoscaling based on metrics such as CPU utilization or request concurrency. By setting a low minimum replica count, you keep baseline cost low during normal periods. A maximum replica count sized for peak load allows the endpoint to scale out automatically when traffic increases tenfold. This elastic behavior handles short bursts without manual intervention and avoids paying for peak capacity all day, which is exactly what the scenario requires.

Answer

Increase the machine type to a larger instance with more memory and vCPUs.

Answer

Deploy the model to a batch prediction job and schedule it to run every hour.

Answer

Enable request logging and set up an alert when latency exceeds a threshold.

Question 14

You are deploying a model to a Vertex AI endpoint for online predictions. You need to ensure that the endpoint can handle traffic spikes and that predictions are served with low latency. Which TWO of the following configurations should you apply? (Choose two.)

Accepted Answer

Configure autoscaling based on CPU utilization with a target utilization that triggers scale-out early.. To handle traffic spikes and ensure low latency, you should maintain a minimum replica count greater than 1 to have warm replicas ready, and configure autoscaling with a low target utilization to trigger scale-out early. These two settings together provide both baseline capacity and proactive scaling, reducing the risk of cold starts and overload during spikes. Other options like logging or larger machines do not directly address dynamic traffic handling.

Answer

Deploy the model to multiple endpoints and use a load balancer to distribute traffic.

Answer

Use a larger machine type with more CPU and memory for each replica to increase per-replica throughput.

Answer

Enable request-response logging to capture detailed latency metrics for each prediction.

Question 15

You are serving a model on a Vertex AI endpoint that requires a GPU. The model is used for interactive predictions with a strict latency SLO. You notice that during peak hours, some requests time out because the endpoint's autoscaler is slow to add GPU replicas. Which action should you take to meet the SLO?

Accepted Answer

Set a higher minimum replica count on the endpoint to keep more GPU replicas warm during peak hours.. GPU replicas take longer to provision and initialize than CPU replicas, so autoscaling may not react quickly enough during sudden peaks. Setting a higher minimum replica count keeps additional GPU replicas warm and ready before demand spikes. This pre-provisioned capacity absorbs peak traffic immediately, preventing request queuing and timeouts. It addresses the slow autoscaler response directly, ensuring the endpoint can meet its strict latency SLO during peak hours.

Answer

Enable request batching on the endpoint to process multiple requests per replica.

Answer

Reduce the model's input size by truncating features to lower per-request compute.

Answer

Switch to a CPU-based machine type to avoid GPU provisioning delays.