PMLE Practice Test 7 — 10 Questions

Question 1

A team uses Vertex AI Pipelines with CustomJob components that pull training code from a Cloud Source Repository. The pipeline fails with a 'Permission denied' error when trying to access the repository. The service account used by the pipeline has the 'Source Repository Viewer' role. What is the likely issue?

Accepted Answer

The 'Source Repository Viewer' role is insufficient; the service account needs 'Source Repository Reader' or higher. The 'Source Repository Viewer' role only allows listing and viewing repository metadata, not reading the actual source code. To clone or pull code from a Cloud Source Repository, the service account needs the 'Source Repository Reader' role (or higher), which grants the `source.repos.get` and `source.repos.read` permissions required for Git operations. The pipeline's CustomJob component fails because the service account lacks these permissions when attempting to access the repository.

Answer

The training code contains a dependency that is not available in the custom container

Answer

The pipeline is running in a different project than the repository; cross-project access is not supported

Answer

The repository URL is incorrectly formatted; use the SSH URL instead of HTTPS

Question 2

A data science team deploys a PyTorch model using Vertex AI Prediction. The model requires GPU for inference, but they notice high costs and underutilized GPUs during off-peak hours. What is the most cost-effective solution?

Accepted Answer

Use a GPU instance with min replicas=0 and autoscaling. Option C is correct because setting min replicas to 0 allows Vertex AI Prediction to scale down to zero instances during off-peak hours, eliminating GPU costs when no requests are being served. Combined with autoscaling, the deployment will spin up GPU-backed instances on demand only when traffic arrives, directly addressing the underutilization issue while maintaining low latency for inference requests.

Answer

Move the model to Cloud Functions

Answer

Use a GPU instance with a fixed number of replicas

Answer

Switch to a CPU-only machine type

Question 3

A company serves a scikit-learn model on Vertex AI Prediction but receives a 400 error with 'Prediction failed: Model evaluation error'. What is the most likely cause?

Accepted Answer

The model uses a scikit-learn version not supported by Vertex AI. Vertex AI Prediction supports specific versions of scikit-learn for serving models. If the model was trained with a version that is not in the supported list (e.g., 0.19, 0.20, 0.22, 0.23, 0.24, 1.0, 1.1), the prediction endpoint will fail with a 'Model evaluation error' because the underlying runtime cannot load the serialized model (e.g., pickle or joblib file). This is the most likely cause of a 400 error when the input format is otherwise correct.

Answer

The input data format is incorrect

Answer

The model was trained with a different framework

Answer

The endpoint is overloaded and timing out

Question 4

A company wants to serve a large XGBoost model that exceeds the 2GB limit for Vertex AI Prediction. What should they do?

Accepted Answer

Use a custom container to serve the model. Vertex AI Prediction has a 2GB limit for the model artifact when using pre-built containers. A custom container bypasses this limit because you package the model and serving code into a Docker image, which can be arbitrarily large. This allows you to serve XGBoost models exceeding 2GB without size constraints imposed by the managed serving infrastructure.

Answer

Reduce model size by removing features

Answer

Compress the model using gzip and upload

Answer

Deploy the model on Cloud Run Functions

Question 5

A company wants to reduce costs for serving a model on Vertex AI Prediction without sacrificing availability. Which THREE strategies should they consider?

Accepted Answer

Enable automatic batching to improve throughput per instance. Option C is correct because enabling automatic batching on Vertex AI Prediction allows the model server to group multiple inference requests into a single batch, which increases throughput per instance and reduces the total number of compute resources needed. This directly lowers serving costs without sacrificing availability, as the batching is handled transparently by the Vertex AI Prediction infrastructure.

Answer

Use larger machine types to reduce the number of replicas

Answer

Switch to HTTP/2 to reduce network overhead

Question 6

Your company uses a custom container for model serving on Vertex AI. After a recent update, the model returns predictions but they are clearly wrong (e.g., negative probabilities for a classification model). The logs show no errors. What is the most likely cause?

Accepted Answer

The preprocessing code in the container was updated but the model was not retrained on the new preprocessing. Option A is correct because the most likely cause of a model returning predictions without errors, but with clearly wrong outputs like negative probabilities, is a mismatch between the preprocessing logic used during training and inference. If the preprocessing code in the container was updated (e.g., scaling, normalization, or feature engineering steps changed) but the model was not retrained on data processed with that new logic, the model receives inputs that are out of distribution, leading to nonsensical outputs. Vertex AI containers run inference with the deployed code, so any change in preprocessing directly affects the input tensor values without raising runtime errors.

Answer

The model file is corrupted

Answer

The model file was accidentally replaced with a different model

Answer

The container is using an incompatible version of the serving framework

Question 7

You are monitoring a machine learning pipeline that runs on Vertex AI Pipelines. The pipeline occasionally fails with a 'ResourceExhausted' error when attempting to read data from BigQuery. Which action should you take to resolve this issue?

Accepted Answer

Reduce the complexity of the BigQuery query or increase the reservation size. The 'ResourceExhausted' error when reading from BigQuery indicates that the query is consuming more resources than the BigQuery reservation allows. Option C is correct because reducing query complexity (e.g., using fewer JOINs, aggregations, or partitions) or increasing the reservation size directly addresses the root cause by either lowering resource demand or allocating more capacity. Other options like switching to Cloud Storage or adjusting pipeline memory do not fix the BigQuery-specific quota or slot exhaustion.

Answer

Switch from BigQuery to Cloud Storage for data source

Answer

Increase the memory allocated to the pipeline step

Answer

Reduce the batch size of the data being read

Question 8

An e-commerce company uses a recommendation model that suggests products based on user browsing history. The model was trained on data from the past year and has high accuracy on the test set. However, after deployment, the click-through rate (CTR) on recommendations is much lower than expected. Which three steps should the data scientist take to diagnose and improve the model? (Choose THREE)

Accepted Answer

Set up an A/B experiment comparing the model's recommendations against a baseline. Option B is correct because an A/B experiment directly measures the model's real-world impact by comparing its CTR against a baseline (e.g., random or popularity-based recommendations). This isolates the model's performance from confounding factors like seasonality or user behavior changes, providing a causal estimate of its effectiveness.

Answer

Run offline evaluation on a holdout dataset to confirm accuracy

Answer

Increase the training dataset size by including data from two years ago

Question 9

Which TWO metrics should you monitor to detect data drift in a batch prediction pipeline?

Accepted Answer

Feature distribution drift (e.g., KS test). Feature distribution drift (C) is correct because it directly measures changes in the input data distribution over time using statistical tests like the Kolmogorov-Smirnov (KS) test, which compares the cumulative distribution of a feature in the current batch against a reference baseline. This is a primary indicator of data drift, as shifts in feature distributions can degrade model performance even if labels are not immediately available.

Answer

Model accuracy on recent labeled data

Answer

Model prediction latency

Answer

Training data size

Question 10

Which THREE components should you include in a comprehensive model monitoring dashboard for a production ML system?

Accepted Answer

System resource utilization (CPU, memory, latency). Option B is correct because system resource utilization metrics (CPU, memory, latency) are essential for monitoring the health and performance of the production infrastructure hosting the ML model. These metrics help detect resource bottlenecks, scaling issues, or degradation that could impact inference latency and throughput, which are critical for maintaining service-level objectives (SLOs).

Answer

Team member roles and responsibilities

Answer

Training pipeline code version