PMLE Practice Exam 63 — 20 Questions

Question 1

A team has a prototype image classification model trained on a small dataset using TensorFlow Keras on a single GPU. They need to train on a larger dataset (1 million images) using a distributed strategy on Vertex AI with 8 GPUs. They implement a MirroredStrategy for data parallelism. During the first few epochs, the training speed does not improve significantly compared to a single GPU, and GPU utilization is low. The data is stored as JPEG files in Cloud Storage, and the input pipeline uses tf.data with map to decode images. What is the most likely cause?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The data loading from Cloud Storage is a bottleneck.. Reading and decoding JPEG images from Cloud Storage can be I/O-bound, causing low GPU utilization. Even though MirroredStrategy is used, if the input pipeline cannot supply data fast enough, GPUs will spend time waiting for data. Option A is incorrect because a large batch size per GPU typically increases memory usage and computational load, not decreases utilization; it might even improve utilization if the model fits. Option B is incorrect because MirroredStrategy is a straightforward configuration for synchronous data parallelism and does not require complex tuning; it is likely configured correctly. Option D is incorrect because the model size does not directly cause low GPU utilization; even a small model can saturate GPUs if the data pipeline is efficient. The primary bottleneck here is the data loading from Cloud Storage.

Answer

The batch size per GPU is too large.

Answer

The MirroredStrategy is not properly configured.

Answer

The model is too small for distributed training.

Question 2

A data scientist trained a model on historical data from 2020-2022 and deployed it in January 2023. In February 2023, the model's accuracy drops significantly. Which monitoring metric would most likely indicate the root cause?

Accepted Answer

Training-serving skew detected by Vertex AI Model Monitoring.. Vertex AI Model Monitoring specifically detects training-serving skew, which occurs when the distribution of input features at serving time differs from the training data distribution. Since the model was trained on 2020-2022 data and deployed in January 2023, a significant accuracy drop in February 2023 likely indicates that the real-world data distribution has shifted (e.g., seasonal patterns, new user behavior), causing the model to encounter unseen patterns. This skew is a common root cause of performance degradation and is directly monitored by Vertex AI's skew detection feature.

Answer

Number of unique users calling the endpoint.

Answer

Prediction latency p99.

Answer

Number of missing feature values in requests.

Question 3

A company wants to reduce costs for serving a model on Vertex AI Prediction without sacrificing availability. Which THREE strategies should they consider?

Accepted Answer

Enable automatic batching to improve throughput per instance. Enabling automatic batching on Vertex AI Prediction allows the model server to group multiple inference requests into a single batch, which increases throughput per instance and reduces the total number of compute resources needed. This directly lowers serving costs without sacrificing availability, as the batching is handled transparently by the Vertex AI Prediction infrastructure.

Answer

Use larger machine types to reduce the number of replicas

Answer

Switch to HTTP/2 to reduce network overhead

Question 4

An ML team is designing an automated pipeline to retrain a recommendation model every day using new user interaction data stored in BigQuery. The pipeline must be cost-efficient, scalable, and require minimal manual intervention. Which two approaches should they consider?

Accepted Answer

Use Cloud Scheduler to publish a Pub/Sub message daily, which triggers a Cloud Function that starts the Vertex AI Pipeline.. Cloud Scheduler triggers a Pub/Sub message that invokes a Cloud Function, which starts a Vertex AI Pipeline. This serverless approach is cost-efficient (no idle compute), scales automatically, and requires minimal manual intervention. Option E is correct because Vertex AI Pipelines natively orchestrates ML workflows, and using preemptible VMs reduces training costs by up to 80% while maintaining scalability.

Answer

Deploy a custom Kubernetes cron job on GKE to run the training script directly.

Answer

Use Cloud Composer (Airflow) to schedule the pipeline with a DAG.

Answer

Use Dataflow to continuously read from BigQuery and trigger training when new data arrives.

Question 5

You have a Vertex AI endpoint with min_replica_count=2 and max_replica_count=10. You notice that during a traffic spike, the endpoint does not scale up quickly enough, causing increased latency. What should you do to improve autoscaling responsiveness?

Accepted Answer

Reduce the target CPU utilization percentage from default to a lower value.. Reducing the target CPU utilization percentage (e.g., from the default 60% to a lower value like 40%) causes the autoscaler to trigger scale-up actions sooner, as the threshold for adding replicas is reached at a lower CPU load. This improves responsiveness during traffic spikes by initiating scaling earlier, reducing latency. The endpoint's min_replica_count=2 and max_replica_count=10 remain unchanged, so the scaling range is preserved.

Answer

Increase max_replica_count to 20.

Answer

Disable autoscaling and manually manage replicas.

Answer

Increase min_replica_count to 10.

Question 6

Refer to the exhibit. A data scientist notices that predictions from a deployed model are taking longer than expected. Which Cloud Monitoring metric should be inspected first to identify the bottleneck?

Accepted Answer

Vertex AI - Endpoint - Prediction latency distribution. The data scientist is investigating slow predictions from a deployed model. The most direct metric to identify the latency bottleneck is the prediction latency distribution, which shows the distribution of response times for online prediction requests. This metric allows you to pinpoint whether the delay is due to model inference time, network overhead, or endpoint queuing, making it the first logical place to inspect.

Answer

Vertex AI - Model - Compute utilization

Answer

Vertex AI - Endpoint - Traffic

Answer

Vertex AI - Endpoint - Online prediction errors

Question 7

A company deploys a TensorFlow model on Vertex AI Prediction with a single node. During peak hours, inference latency increases. What should they do first to reduce latency?

Accepted Answer

Enable autoscaling for the deployment. Enabling autoscaling for the deployment is the correct first step because it allows Vertex AI Prediction to dynamically adjust the number of replicas based on incoming traffic. During peak hours, autoscaling can add more nodes to distribute the inference load, directly reducing latency without requiring manual intervention or over-provisioning.

Answer

Increase the machine type of the node

Answer

Decrease the min replicas to 0

Answer

Enable automatic batching of requests

Question 8

An ML engineer needs to monitor a deployed model for data drift. They want to compare the distribution of incoming predictions against a baseline distribution. Which Vertex AI service should they use?

Accepted Answer

Vertex AI Model Monitoring. Vertex AI Model Monitoring is the correct service because it is specifically designed to detect data drift and feature skew in deployed models. It continuously compares the distribution of incoming prediction requests against a baseline distribution (e.g., training data or a previous window) and alerts the engineer when statistically significant drift is detected, using metrics like Jensen-Shannon divergence or L-infinity distance.

Answer

Vertex AI Feature Store

Answer

Vertex AI Experiments

Answer

Vertex AI Explainable AI

Question 9

A machine learning engineer is training a large-scale text classification model using a distributed strategy on TPUs. The training loss decreases normally but the validation loss starts increasing after a few epochs while training loss continues to decrease. The engineer suspects overfitting. Which technique is most appropriate to address this while scaling training?

Accepted Answer

Add dropout regularization.. Dropout regularization is a widely used technique to prevent overfitting in neural networks by randomly dropping units during training, which forces the model to learn more robust features. It can be applied in distributed training on TPUs without major modifications and directly addresses overfitting by reducing co-adaptation of neurons. Option B is wrong because early stopping halts training when validation loss increases, but it does not prevent the underlying overfitting from occurring during training. Option C is wrong because reducing the learning rate controls the step size of gradient updates and is primarily used for convergence issues, not directly for overfitting. Option D is wrong because increasing batch size can sometimes improve generalization due to reduced gradient noise, but it is not a primary or most appropriate method to combat overfitting; it may even hurt generalization in some cases.

Answer

Use early stopping with patience.

Answer

Reduce the learning rate.

Answer

Increase the batch size.

Question 10

A company is using AutoML Vision for object detection and observes high latency for online predictions. What can they do to reduce latency?

Accepted Answer

Reduce the training budget to create a smaller model. Reducing the training budget in AutoML Vision forces the model to use fewer node-hours, which typically results in a smaller and less complex model. A smaller model has fewer parameters and requires less computation during inference, directly reducing the latency for online predictions. This is a trade-off between model accuracy and inference speed.

Answer

Use continuous batch prediction instead of online prediction

Answer

Deploy the model to a region closer to the users

Answer

Use a larger batch size in the prediction request

Question 11

A company uses Vertex AI Pipelines to orchestrate their ML training workflow. The pipeline includes a BigQuery ML training step, a model evaluation step, and a deployment step to Vertex AI Endpoints. The engineer notices that the pipeline fails intermittently due to a quota exceeded error on Vertex AI Endpoints during model deployment. What is the best long-term solution to prevent this failure?

Accepted Answer

Add retry logic with exponential backoff to the deployment step in the pipeline.. Adding retry logic with exponential backoff handles transient quota errors gracefully without manual intervention, making it a resilient and long-term solution for intermittent failures. Option A is wrong because longer wait times do not address the root cause of quota exhaustion. Option C is wrong because deploying on Compute Engine introduces additional complexity and does not resolve Vertex AI quota issues. Option D is wrong because requesting a permanent quota increase may be necessary for sustained high volume, but it requires planning and approval, and is not the best first step for intermittent errors.

Answer

Run the pipeline steps sequentially with longer wait times.

Answer

Switch to deploying models using a custom container on Compute Engine.

Answer

Request a permanent quota increase for Vertex AI Endpoints.

Question 12

You are using Vertex AI to train a model with a custom container. You need to pass command-line arguments for hyperparameters. Which TWO methods can you use? (Choose 2.)

Accepted Answer

Use the 'command' field to override the entrypoint and include arguments.. Options D and E are correct because the 'command' field allows you to override the container's entrypoint and include command-line arguments, and the 'args' field appends arguments to the entrypoint or command. Option B is incorrect because environment variables are not command-line arguments; they are a separate mechanism for passing configuration. Options A and C are also incorrect.

Answer

Store arguments in a Cloud Storage file and download at runtime.

Answer

Set environment variables using the 'env' field and reference them in the container.

Answer

Set hyperparameter values in the 'hyperparameters' field of the worker pool spec.

Question 13

Your team has deployed a model on Vertex AI endpoints. You need to monitor the prediction latency to ensure it meets a 99th percentile SLO of 500ms. You want to set up an alert if the latency exceeds this threshold. Which metric should you use?

Accepted Answer

The 99th percentile of the `prediction/online/response_latencies` metric.. The `prediction/online/response_latencies` metric in Vertex AI provides a distribution of latency values, allowing you to query the 99th percentile directly. This aligns with the SLO requirement to monitor the tail latency, not the average or maximum, ensuring that the worst-case performance for 1% of requests stays under 500ms.

Answer

The number of prediction requests that timeout.

Answer

Average prediction latency from the endpoint's logs.

Answer

The maximum prediction latency from the endpoint's monitoring dashboard.

Question 14

A company serves a scikit-learn model on Vertex AI Prediction but receives a 400 error with 'Prediction failed: Model evaluation error'. What is the most likely cause?

Accepted Answer

The model uses a scikit-learn version not supported by Vertex AI. Vertex AI Prediction supports specific versions of scikit-learn for serving models. If the model was trained with a version that is not in the supported list (e.g., 0.19, 0.20, 0.22, 0.23, 0.24, 1.0, 1.1), the prediction endpoint will fail with a 'Model evaluation error' because the underlying runtime cannot load the serialized model (e.g., pickle or joblib file). This is the most likely cause of a 400 error when the input format is otherwise correct.

Answer

The input data format is incorrect

Answer

The model was trained with a different framework

Answer

The endpoint is overloaded and timing out

Question 15

A team is building a CI/CD pipeline for ML using Cloud Build. The pipeline trains a model and deploys it to Vertex AI. Recently, a change in the data processing step caused the model to be trained with a different data version, leading to a failed deployment because the model was invalid. How should the team prevent this in the future?

Accepted Answer

Use a data versioning tool (e.g., DVC) to track datasets and ensure the pipeline always uses the correct version. The root cause is a data version mismatch, not a code or environment issue. A data versioning tool like DVC (Data Version Control) tracks dataset versions via hash-based pointers in Git, ensuring the pipeline retrieves the exact dataset version used during training. This prevents silent failures when data processing steps change the data schema or content, which library pinning or manual reviews cannot guarantee.

Answer

Add a manual review step before training

Answer

Pin all library versions in the Docker image

Answer

Schedule a cron job to check for data changes

Question 16

A team is monitoring a production ML system that includes multiple models and data processing pipelines. They want to set up a comprehensive alerting strategy that minimizes false positives while ensuring critical issues are promptly addressed. Which approach is the most effective?

Accepted Answer

Use AIOps with anomaly detection to dynamically adjust thresholds. AIOps with anomaly detection uses machine learning to dynamically adjust alert thresholds based on real-time system behavior, reducing false positives while ensuring critical issues are detected promptly. This approach adapts to changing data distributions and traffic patterns, unlike static thresholds that require manual tuning and often miss subtle anomalies. It is the most effective strategy for complex ML production systems where multiple models and pipelines interact, as it can correlate signals across components to identify genuine incidents.

Answer

Set up alerts for all possible error conditions

Answer

Use static thresholds based on historical data

Answer

Rely on manual monitoring during business hours

Question 17

A company has a pipeline that uses Vertex AI Pipelines to fetch data from BigQuery, preprocess with Dataflow (without code?), then train an AutoML model, and deploy. However, they want to reduce cloud costs. The pipeline runs hourly. Which change will most reduce compute costs while maintaining throughput?

Accepted Answer

Increase Dataflow batch size to process more data per worker. Increasing the Dataflow batch size allows each worker to process more data per batch, reducing the number of workers needed and the total compute time for the same throughput. This directly lowers Dataflow's compute cost without affecting the pipeline's hourly schedule or the AutoML training budget.

Answer

Decrease the AutoML training budget from 10 to 1 node hour

Answer

Replace Dataflow preprocessing with a Cloud Function that runs on each file upload

Answer

Switch from Vertex AI Pipelines to Cloud Composer for orchestration

Question 18

An ML engineer is creating a Vertex AI Pipeline that includes a loop to train multiple models in parallel on different hyperparameter sets. Which TWO KFP SDK v2 constructs can be used to implement this parallel execution?

Accepted Answer

A for loop in the pipeline function that creates multiple tasks. In KFP SDK v2, you can use a Python for loop inside the pipeline function to dynamically create multiple task instances, each with different hyperparameter sets. These tasks are then executed in parallel by the Vertex AI Pipelines orchestrator, as long as there are no data dependencies between them. This pattern leverages the SDK's ability to compile Python control flow into a directed acyclic graph (DAG) of pipeline steps.

Answer

dsl.If

Answer

dsl.Parallel

Answer

dsl.Pipeline

Question 19

A machine learning engineer is exporting a trained model from Vertex AI Training to the Model Registry. Which artifact should they upload as the model artifact?

Accepted Answer

The saved model directory containing the model file(s) and any custom dependencies.. When exporting a trained model from Vertex AI Training to the Model Registry, the correct artifact is the saved model directory that contains the model file(s) (e.g., SavedModel format for TensorFlow, model.pkl for scikit-learn) along with any custom dependencies required for serving. This ensures the model can be deployed consistently to endpoints or batch predictions, as the Model Registry expects a self-contained artifact that includes both the model binary and its runtime dependencies.

Answer

Only the model checkpoint file (.ckpt or .h5).

Answer

The entire training directory including training code and logs.

Answer

A zip file of the training source code.

Question 20

A team wants to collect ground truth labels for their model deployed on Vertex AI Endpoint to perform model quality monitoring. They have a process that generates actual outcomes within 24 hours of prediction. What is the recommended approach for storing these labels?

Accepted Answer

Upload the ground truth labels to a BigQuery table with a schema that includes prediction timestamp and model version.. Vertex AI Model Monitoring for model quality requires ground truth data to be uploaded to BigQuery. Labels are stored in a BigQuery table with the prediction timestamp and model ID for comparison.

Answer

Use Vertex AI Experiments to log ground truth alongside training runs.

Answer

Store the ground truth labels in Cloud Storage as CSV files and reference them in the monitoring config.

Answer

Insert ground truth labels directly into the Vertex AI Endpoint's log sink.