PMLE Timed Hard Questions — 25 Difficult Questions

Question 1

In a Vertex AI Pipeline, a component produces a Metrics artifact that includes an evaluation metric. The engineer wants to use this metric value as a condition to decide whether to deploy the model. However, the metric value is stored in the artifact's metadata and not directly as a pipeline parameter. How can the engineer pass the metric value to a downstream conditional task?

15 minute time limit — choose an answer to begin.

Accepted Answer

Add a component that reads the artifact's metadata and outputs the metric as a parameter, then use that parameter in the condition.. Vertex AI Pipeline conditions require pipeline parameters (typed values) to evaluate expressions like `dsl.If`. A Metrics artifact's metadata is stored as an artifact property, not a pipeline parameter, so a custom component must read that metadata and output the metric as a parameter. This parameter can then be used in the `dsl.If` condition to control downstream deployment.

Answer

Configure the component that produces the Metrics artifact to also output the metric as a pipeline parameter.

Answer

Use the importer component to convert the artifact into a parameter.

Answer

Use the artifact directly in the dsl.If condition, as artifacts are comparable.

Question 2

You are fine-tuning a large language model (LLM) from Vertex AI Model Garden using a custom dataset. You need to minimize training cost while maintaining reasonable throughput. Which THREE strategies should you combine?

Accepted Answer

Use spot VM instances for training. Spot VM instances are significantly cheaper than on-demand instances, reducing training cost. They can be preempted, but for fine-tuning tasks that can checkpoint and resume, this trade-off is acceptable for cost savings.

Answer

Use full fine-tuning of all model parameters

Answer

Use TPU v4 pods for training

Question 3

A team is using Vertex AI AutoML to train a forecasting model. They need to retrain the model weekly and only if the new week's data significantly changes the data distribution. What is the most efficient way to achieve this?

Accepted Answer

Use Vertex AI Model Monitoring to detect drift and trigger a pipeline. Vertex AI Model Monitoring can be configured to detect data drift on the model's input features, and when drift exceeds a threshold, it can trigger a Cloud Function or a Vertex AI pipeline to retrain the model. This approach avoids unnecessary retraining when the data distribution has not changed significantly, which is more efficient than always retraining. The integration with Cloud Functions or Pub/Sub allows for a serverless, event-driven retraining pipeline that only runs when needed.

Answer

Use a scheduled pipeline that always retrains

Answer

Use Cloud Monitoring alerts on data drift to trigger retraining

Answer

Use Cloud Functions on schedule to compare distributions

Question 4

A machine learning team uses Vertex AI Pipelines to orchestrate their training pipeline. They want to trigger the pipeline automatically in response to new data arriving in a Cloud Storage bucket, and also support a scheduled run every day at 6 AM. Which combination of services should they use to achieve both event-driven and schedule-based triggers?

Accepted Answer

Cloud Scheduler for the schedule, and Cloud Functions triggered by Cloud Storage events to call the Vertex AI API for event-driven.. Cloud Scheduler can trigger the pipeline at 6 AM daily via a cron job, while Cloud Functions, triggered by Cloud Storage events (e.g., object finalize), can call the Vertex AI API to start the pipeline when new data arrives. This combination provides both schedule-based and event-driven triggers without requiring custom infrastructure.

Answer

Cloud Scheduler for the schedule, and Cloud Pub/Sub with Push subscription to Vertex AI for event-driven.

Answer

Cloud Functions for both schedule and event-driven, using cron trigger.

Answer

Vertex AI Pipelines built-in scheduler for schedule, and Cloud Pub/Sub for event-driven.

Question 5

You are using Vertex AI Vector Search for a product recommendation system. Your index is updated with new embeddings every hour. To minimize query latency while keeping the index fresh, what should you do?

Accepted Answer

Use streaming updates to insert new embeddings into the deployed index.. Vertex AI Vector Search supports streaming updates that let you insert, update, or delete datapoints in a deployed index without rebuilding it, so queries stay low-latency while the index remains fresh. This is the intended mechanism for near-real-time freshness with minimal disruption. Batch rebuilds and redeployments cause downtime and latency spikes, which the question explicitly asks to avoid.

Answer

Rebuild the entire index hourly as a batch job and redeploy it.

Answer

Create a new index each hour and use traffic splitting to gradually shift traffic.

Answer

Use a brute-force index instead of ANN to ensure accuracy after updates.

Question 6

A company needs to perform real-time similarity search on a dataset of 10 million embedding vectors. They expect low latency (under 10ms) and high throughput. Which index type should they use in Vertex AI Vector Search?

Accepted Answer

Approximate nearest neighbor (ANN) index with ScaNN. For large datasets requiring low latency, an approximate nearest neighbor (ANN) index is appropriate. The Scann algorithm (ScaNN) is used by Vertex AI Vector Search for ANN.

Answer

Brute-force index

Answer

Hash-based index

Answer

Tree-based index

Question 7

You are using Vertex AI Prediction with a custom container that requires a large model file (5 GB). Deployment takes 10 minutes to start. You want to reduce cold start latency. Which action would be MOST effective?

Accepted Answer

Set minReplicas to 1 to keep at least one instance always running.. Setting minReplicas to 1 keeps at least one prediction instance warm, eliminating the cold start caused by loading the 5 GB model from scratch on each new deployment. Vertex AI Prediction scales replicas based on traffic; with minReplicas=1, the model stays loaded in memory and responds immediately. Compression, local SSD, and batch prediction do not address the fundamental issue of keeping a warm online endpoint.

Answer

Compress the model file and decompress on startup.

Answer

Use a machine type with local SSD to speed up model loading.

Answer

Switch to batch prediction to avoid online cold start.

Question 8

A company has a prototype ML model that predicts equipment failure. They want to deploy it to production using Vertex AI. The model must be retrained weekly with new data. They also need to monitor for data drift and model performance. Which THREE components should they include in their MLOps pipeline? (Choose 3)

Accepted Answer

A scheduled training pipeline that retrains the model weekly.. Option A is correct because the requirement to retrain weekly with new data is met by a scheduled training pipeline in Vertex AI, typically implemented with Vertex AI Pipelines and a Cloud Scheduler or scheduled run that re-executes the training job on the latest data. Option D is correct because automated redeployment when performance drops below a threshold is a core MLOps practice; in Vertex AI this can be done by monitoring model performance and using a trigger (for example, a Cloud Function or Eventarc/Cloud Monitoring alert) to promote a newly trained model to an endpoint. Option E is correct because Vertex AI Model Monitoring detects data drift and can emit alerts via Cloud Monitoring, which is exactly what the scenario requires for drift and performance oversight. Option B is not correct because a manual QA approval step for every deployment is not required by the scenario and would slow down the automated weekly retraining and redeployment workflow. Option C is not correct because manual review of all new data before training is not a required MLOps component here and would prevent the pipeline from being fully automated and scheduled weekly.

Answer

A manual QA step where data scientists approve each deployment.

Answer

A manual review of new data before it is used for training.

Question 9

A machine learning team is deploying a PyTorch model on Vertex AI Prediction for real-time inference. The model was trained with preprocessing that includes tokenization and normalization. They want to embed the preprocessing logic in the model to reduce prediction latency and avoid additional service calls. Which approach should they take?

Accepted Answer

Use TorchScript to trace the preprocessing steps and export the entire pipeline as a single scripted model. TorchScript allows you to trace or script a PyTorch model, including preprocessing operations like tokenization and normalization, into a single serialized artifact. By embedding preprocessing in the TorchScript model, the entire pipeline runs in one forward pass on the Vertex AI endpoint, eliminating extra service calls and reducing latency. This is the standard approach for consolidating preprocessing with a PyTorch model for real-time inference.

Answer

Deploy the preprocessing logic as a Cloud Function and invoke it before calling the prediction endpoint

Answer

Wrap the preprocessing logic in a Flask application and deploy it as a separate microservice in front of the prediction endpoint

Answer

Use TensorFlow Transform to convert preprocessing into a SavedModel and call it from the PyTorch model

Question 10

A financial institution needs to extract structured data from scanned PDFs of loan applications, including text fields and tables. They require a human review step for high-risk applications. Which Google Cloud service and configuration should they use?

Accepted Answer

Document AI with a form parser processor and enable Human-in-the-Loop for high-risk applications. Document AI's Form Parser processor is purpose-built to extract structured key-value pairs and tables from forms like loan applications, and it natively integrates with Human-in-the-Loop (HITL) to route low-confidence or high-risk documents to human reviewers. This combination directly satisfies both the extraction and human review requirements without custom code.

Answer

Document AI with a custom extractor processor and use Cloud Functions for human review

Answer

Cloud Vision API to detect text and tables, then send to Cloud Dataflow for processing

Answer

Vertex AI AutoML Vision to train a custom model for document parsing

Question 11

A data scientist wants to perform A/B testing between two model versions deployed on the same Vertex AI endpoint. They need to route 10% of traffic to the challenger model. Which approach should they use?

Accepted Answer

Deploy both models to the same endpoint and set traffic_split to 90 for champion and 10 for challenger. Vertex AI endpoints support traffic splitting directly, allowing you to route a percentage of requests to different model versions deployed on the same endpoint. By setting `traffic_split` to 90 for the champion and 10 for the challenger, the data scientist can perform online A/B testing without additional infrastructure. This is the simplest and most cost-effective approach, as it avoids managing separate endpoints or load balancers.

Answer

Use Vertex AI Experiments to compare models offline, then deploy the winner

Answer

Deploy the challenger model to a separate endpoint and use a load balancer to split traffic

Answer

Update the champion model with a new version and use model version aliases

Question 12

A company has multiple teams working on different models. They want to enforce consistent data preprocessing steps across all teams. Which approach should they take?

Accepted Answer

Create shared Vertex AI Pipelines components. Vertex AI Pipelines components allow teams to define reusable, versioned, and parameterized preprocessing steps that can be shared across models and pipelines. This ensures consistent execution of data transformations because each component encapsulates the exact code and environment, and pipelines enforce the same DAG of steps regardless of which team triggers them.

Answer

Use Cloud Composer to orchestrate preprocessing

Answer

Write shared Python packages in Artifact Registry

Answer

Use Cloud Dataflow templates

Question 13

A data engineering team needs to compute rolling window features (7-day average, 30-day sum) from a high-volume stream of e-commerce events stored in BigQuery. They must output the features to Vertex AI Feature Store for online serving. Which approach is MOST cost-effective and scalable?

Accepted Answer

Use Dataflow with Apache Beam, reading from BigQuery, computing windowed aggregations, and writing to Vertex AI Feature Store. Dataflow (Apache Beam) is ideal for processing large-scale batch and streaming data. It can read from BigQuery, perform windowing computations, and write to Feature Store's online store. Cloud Functions have timeouts, Cloud Composer is not optimal for streaming, and BigQuery scheduled queries are not designed for streaming-feature computation.

Answer

Use Cloud Composer (Airflow) with a daily DAG to run SQL queries on BigQuery and export results

Answer

Schedule a query in BigQuery using scheduled queries and export results to Feature Store

Answer

Use Cloud Functions triggered by Pub/Sub to compute features on the fly

Question 14

A team is monitoring a batch prediction job on Vertex AI. Which two metrics should they monitor to ensure the job completes successfully without errors?

Accepted Answer

Job failure rate. Option C (Job failure rate) is correct because a batch prediction job's success is directly measured by whether its tasks complete without errors; monitoring the failure rate surfaces failed prediction tasks or job-level errors so the team can detect and remediate problems before the job is considered unsuccessful. Option E (Number of preempted workers) is correct because Vertex AI batch prediction jobs can run on preemptible/Spot VMs, and when those workers are preempted the job may stall, retry, or fail; tracking preemptions lets the team confirm the job has enough stable capacity to finish. Option A (Data size of input) is not a success/error indicator—it is a capacity/cost input characteristic rather than a health metric. Option B (Prediction requests per second) describes throughput of an online prediction workload, not whether a batch job completes without errors. Option D (Model endpoint latency) applies to online prediction endpoints, whereas batch prediction jobs do not serve through a deployed endpoint.

Answer

Data size of input

Answer

Prediction requests per second

Answer

Model endpoint latency

Question 15

A team monitors features in Vertex AI Feature Store for drift. They want to set up automated alerts when a feature's distribution deviates significantly from the baseline. Which feature monitoring configuration should they use?

Accepted Answer

Enable feature monitoring on the feature group with drift threshold and notification channel.. Feature monitoring in Vertex AI Feature Store allows defining drift thresholds and alerting via Cloud Monitoring.

Answer

Use Cloud Monitoring custom metrics and log-based alerts manually.

Answer

Use Vertex AI Experiments to compare distributions.

Answer

Export features to BigQuery and set up scheduled queries with alerts.

Question 16

An ML engineer is building a monitoring dashboard for a Vertex AI pipeline that includes training, evaluation, and batch prediction. Which THREE components should be included to provide comprehensive observability? (Select THREE.)

Accepted Answer

Pipeline execution status, duration, and failure rates for each component.. Option A is correct because tracking pipeline execution status, duration, and failure rates per component gives the operational observability needed to detect where a training, evaluation, or batch prediction step fails or slows down. Option C is correct because model evaluation metrics such as accuracy and AUC after training and validation are essential to monitor model quality and detect degradation before deployment. Option D is correct because data validation reports with anomaly counts and feature statistics surface input data drift, skew, and schema issues that directly affect pipeline and model reliability. Option B is not appropriate because raw Compute Engine CPU and memory logs for each step are low-level infrastructure telemetry, not the pipeline-level observability components the scenario requires. Option E is not appropriate because online prediction latency and request counts apply to a deployed endpoint, whereas this pipeline uses batch prediction and has no online serving endpoint.

Answer

Compute engine CPU and memory logs for each pipeline step.

Answer

Online prediction latency and request count from the deployed model endpoint.

Question 17

A machine learning team wants to implement a continuous delivery pipeline for their ML models using Vertex AI Pipelines. The pipeline should automatically deploy a model to a staging endpoint after evaluation passes, and then after manual approval, promote it to production. Which strategy should they use to manage model versions in the Vertex AI Model Registry?

Accepted Answer

Use Vertex AI Model Registry aliases: assign the model version an alias 'staging' initially, and after approval, change the alias to 'production' via the API.. Vertex AI Model Registry supports versioning and alias management. By uploading models with an alias like 'staging' and later changing the alias to 'production', the team can control promotion. The other options either lack the required functionality or add unnecessary complexity.

Answer

Store model artifacts in Cloud Storage with versioned paths and deploy directly from Storage without using the Model Registry.

Answer

Create two separate models in the registry: one for staging and one for production, and copy artifacts between them after approval.

Answer

Upload each model version with the same model ID and use labels to differentiate staging vs production.

Question 18

An ML engineer is trying to upload a TensorFlow model to Vertex AI using the gcloud command shown. The model was trained using TensorFlow 2.11 and saved with model.save('model/'). The engineer sees the error. What is the most likely cause?

Accepted Answer

The model was saved in a format other than SavedModel (e.g., HDF5) or the artifact path does not contain the expected directory structure.. The error indicates that Vertex AI cannot find the expected SavedModel artifacts (saved_model.pb and variables/ directory) at the specified path. When using model.save('model/') with TensorFlow 2.11, the default format is the SavedModel format, but the artifact path must point to the directory containing the saved_model.pb file, not a parent directory or a model saved in HDF5 format. The gcloud command likely references a path that does not contain the required SavedModel structure, causing the upload to fail.

Answer

The container port should be 8080 instead of 8501.

Answer

The service account does not have permission to access the bucket.

Answer

The container image is for TensorFlow 2.11 but the model was saved with an older version.

Question 19

A team of ML engineers is building a real-time fraud detection system. They use Cloud Pub/Sub to stream transactions, Dataflow for feature engineering, and Vertex AI to get predictions. They want to ensure that the data used for training matches the data used for serving to avoid training-serving skew. Which approach should they take?

Accepted Answer

Use Vertex AI Feature Store to store features computed during training and retrieve them in the serving pipeline.. Vertex AI Feature Store ensures that the same feature engineering logic is applied consistently during both training and serving. By storing precomputed features in the Feature Store, the serving pipeline retrieves the exact same feature values that were used during training, eliminating the risk of training-serving skew. This approach is specifically designed for real-time systems where streaming data (via Pub/Sub and Dataflow) must be served with identical transformations.

Answer

Use a batch processing system for both training and serving to ensure identical feature calculations.

Answer

Implement separate feature engineering pipelines for training and serving, but document them carefully.

Answer

Ensure that both training and serving read from the same Cloud Storage location.

Question 20

A data science team has trained a TensorFlow model on-premises using a large dataset. When they try to deploy the model to Vertex AI for online predictions, the deployed model fails to start with a ‘MemoryError’. The model artifact is 2 GB, and the machine type is n1-standard-4 (15 GB RAM). What is the most likely cause?

Accepted Answer

The model is too large for the machine's memory, causing an out-of-memory (OOM) error during loading.. The model artifact is 2 GB, and loading it into memory on an n1-standard-4 machine (15 GB RAM) can still cause a MemoryError. TensorFlow models often require additional memory for graph construction, intermediate tensors, and framework overhead, which can easily exceed the available RAM, especially when the model is loaded entirely into memory before serving.

Answer

The model is stored in a regional bucket and the Vertex AI endpoint is in a different region.

Answer

The machine type does not support TensorFlow models larger than 1 GB.

Answer

The model file is corrupted or missing dependencies, causing a crash.