PMLE Practice Test 1 — 30 Questions

Question 1

A data science team deploys a regression model to predict house prices. After one month, the mean absolute error (MAE) on the serving data increases by 20% compared to the test set. Which monitoring strategy should the team implement first to diagnose the issue?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Monitor prediction residuals and compute serving-time MAE over sliding windows.. The first step in diagnosing a 20% MAE increase on serving data is to monitor prediction residuals over sliding windows. This directly tracks how model errors evolve in production, allowing the team to detect whether performance degradation is sudden or gradual, and to correlate it with specific time windows or data slices. Computing serving-time MAE on sliding windows provides an immediate, interpretable signal of model health without assuming the root cause.

Answer

Retrain the model daily with the latest data to adapt to changing patterns.

Answer

Compare the distribution of training labels with serving labels using a two-sample t-test.

Answer

Monitor input feature distributions for drift using the Kolmogorov-Smirnov test.

Question 2

You are an ML engineer at a logistics company. You have deployed a deep learning model on Vertex AI Endpoints using a custom container with GPU acceleration. The model predicts delivery times based on route features. After one week, you notice that the endpoint's GPU utilization is consistently at 10%, but the prediction latency has increased by 50%. The number of prediction requests per second has remained stable. You check the container logs and see no errors. The model is served using TensorFlow Serving with batching enabled (batch size: 32, batch timeout: 100ms). The custom container uses a single NVIDIA T4 GPU. You have also set the Vertex AI endpoint to use autoscaling with minReplicaCount: 1 and maxReplicaCount: 5, and the CPU utilization target is 60%. Which action should you take to reduce latency?

Accepted Answer

Increase the batch size to 64 and batch timeout to 200ms to improve GPU utilization.. The core issue is low GPU utilization (10%) despite increased latency, indicating that the GPU is underutilized and the bottleneck is likely in batching or data pipeline overhead. Increasing the batch size to 64 and batch timeout to 200ms allows TensorFlow Serving to accumulate more requests per batch, improving GPU throughput and reducing per-request latency by better leveraging GPU parallelism. This directly addresses the mismatch between low GPU utilization and high latency.

Answer

Increase the minReplicaCount to 3 to handle requests in parallel.

Answer

Reduce the CPU utilization target to 40% to trigger more aggressive autoscaling.

Answer

Quantize the model to FP16 to reduce compute time per inference.

Question 3

Your company runs a high-traffic web application that serves the same machine learning model prediction for many identical requests (e.g., product recommendations for the same user profile). You want to reduce latency and load on the prediction endpoint by caching responses. Which Google Cloud service should you use?

Accepted Answer

Cloud Memorystore. Cloud Memorystore (B) is correct because it provides a managed in-memory cache (Redis or Memcached) that can store the results of identical prediction requests, reducing latency and load on the prediction endpoint. By caching responses keyed on the user profile or request parameters, subsequent identical requests can be served directly from Memorystore in microseconds, avoiding redundant model inference.

Answer

Cloud CDN

Answer

Cloud Spanner

Answer

BigQuery

Question 4

Match each model evaluation metric to its use case.

Accepted Answer

Accuracy: Correct when classes are balanced and errors have equal cost. Accuracy is correctly matched with balanced classes and equal error cost; Precision with minimizing false positives; Recall with minimizing false negatives; F1 Score with balancing precision and recall for imbalanced classes. Option E incorrectly pairs Accuracy with 'when false positives are costly' which is actually the domain of Precision. The main trap is confusing accuracy with precision when dealing with asymmetric costs.

Answer

Accuracy: When false positives are costly

Question 5

A company uses Vertex AI Model Monitoring. Which two configuration options can be set to reduce false positive drift alerts?

Accepted Answer

Use a sample percentage of predictions. Using a sample percentage of predictions reduces the volume of data analyzed for drift, which lowers the chance of detecting statistically insignificant fluctuations that could trigger false positive alerts. This is a common technique to filter out noise in high-throughput production systems.

Answer

Set a shorter alerting window

Answer

Decrease the drift threshold

Answer

Enable feature attribution monitoring

Question 6

Which TWO actions should be taken to ensure reproducibility of ML experiments when collaborating across teams on Vertex AI?

Accepted Answer

Lock dependency versions in a container image used for training. Locking dependency versions in a container image ensures that the exact same software environment (e.g., Python packages, CUDA libraries, system tools) is used every time a training job runs. This eliminates variability from package updates or OS patches, which is a fundamental requirement for reproducibility across teams. Vertex AI supports custom containers for training, making this a direct and reliable method.

Answer

Share notebooks via Colab Enterprise with real-time editing

Answer

Allow each team to use their own preferred environment

Answer

Always use random seeds for all random operations

Question 7

Which TWO strategies can help reduce the cost of running ML pipelines on Vertex AI?

Accepted Answer

Use Vertex AI managed datasets to reduce storage costs. Options B and D are correct. Option B is correct because using Vertex AI managed datasets can reduce storage costs by eliminating duplicate copies and providing efficient storage. Option D is correct because preemptible VMs are significantly cheaper than regular VMs, making them cost-effective for fault-tolerant training tasks. Option A is incorrect because hyperparameter tuning with a large search space can increase cost due to many trials. Option C is incorrect because manual scaling is less efficient and can lead to higher costs compared to automated scaling. Option E is incorrect because using a larger machine type for training usually increases cost, even if it reduces training time.

Answer

Run hyperparameter tuning jobs with a large search space

Answer

Manually scale up resources during peak times and scale down during off-peak

Answer

Use a larger machine type for training to complete faster

Question 8

A company has developed a prototype fraud detection model using a small sample of transactions. The prototype runs on a single VM and uses a Random Forest classifier. They want to scale to the full dataset of 50 million transactions. The data is stored in BigQuery. The team wants to use Vertex AI for training. After moving the code to a custom training container and using Vertex AI Training with a single n1-standard-4 machine, the training job fails with an error: "Process terminated with exit code 1". The logs show: "java.lang.OutOfMemoryError: Java heap space". The model uses a scikit-learn RandomForest. Which course of action is most appropriate?

Accepted Answer

Increase the machine type to n1-highmem-8 to provide more memory.. Option B because increasing memory (n1-highmem-8) directly addresses the Java heap space error. Random Forest memory usage scales with data size, so more memory is needed. Option A is not appropriate because scikit-learn does not natively support distributed training, and setting up distributed Random Forest is complex. Option C is unnecessary; switching to a linear model may reduce performance. Option D is wrong because high-CPU machines have less memory per core, which does not help with memory issues.

Answer

Use a distributed training strategy with multiple workers.

Answer

Switch from Random Forest to a linear model to reduce memory usage.

Answer

Switch to a high-CPU machine type like n1-highcpu-16.

Question 9

An ML team is converting a prototype model to a production pipeline using Vertex AI. They want to ensure model versioning and lineage. Which two practices should they adopt? (Select TWO)

Accepted Answer

Use Vertex AI Model Registry to manage model versions.. Options A and C are correct. Using Vertex AI Model Registry allows you to manage and track model versions, while storing model artifacts in Cloud Storage with unique versioned directories ensures lineage and reproducibility. Option B is wrong because keeping only the latest version loses the history and lineage. Option D is wrong because training without tracking violates versioning principles. Option E is wrong because using separate GCP projects per version is unnecessarily complex and does not directly address versioning and lineage.

Answer

Only keep the latest model version to save storage.

Answer

Train models directly in production without tracking.

Answer

Use a separate GCP project for each model version.

Question 10

A data science team deploys a large language model (LLM) on Vertex AI Prediction using an NVIDIA A100 GPU. The end-to-end latency is acceptable, but the cost is high due to low GPU utilization. The model is stateless and requests are independent. Which strategy would most effectively reduce cost per prediction?

Accepted Answer

Implement dynamic batching in the serving container to aggregate multiple requests into a single inference call.. Dynamic batching aggregates multiple independent requests into a single inference call, increasing GPU utilization and reducing cost per prediction without significantly impacting latency if batch size is tuned. Option A is wrong because migrating to TPUs would require model changes and may not improve GPU utilization. Option B is wrong because using a smaller GPU (T4) may increase latency or reduce throughput, and increasing replicas adds cost. Option C is wrong because reducing min replicas to 0 causes cold starts, increasing latency for each request.

Answer

Migrate the model to Cloud TPU using TensorFlow to benefit from higher throughput.

Answer

Use a smaller GPU, such as NVIDIA T4, and increase the number of replicas to maintain throughput.

Answer

Reduce the number of min replicas to 0 and scale from 0 on each request.

Question 11

A retail company deployed a demand forecasting model using TensorFlow on Vertex AI Batch Prediction. The model runs weekly on a large dataset stored in BigQuery. Over the past month, the prediction accuracy has degraded significantly. The ML engineer reviews the monitoring dashboard and sees that the feature distribution for 'product_price' has shifted from a mean of $50 to $55, and the new product category 'electronics' now represents 20% of the data, whereas it was only 5% in training. The model was never retrained after initial deployment six months ago. The engineer also notices that the Vertex Explainable AI feature importance scores have changed: 'product_price' used to be the top feature (importance 0.35) but now ranks third (importance 0.20). The company requires minimal downtime and wants to improve accuracy as quickly as possible without incurring high costs from excessive retraining. Which course of action should the ML engineer take?

Accepted Answer

Retrain the model using the most recent 3 months of data, including all new product categories, and deploy the updated model via a new Vertex AI endpoint.. The correct action is to retrain the model with the latest data because the feature distributions and data composition have changed significantly, indicating covariate shift and concept drift. Retraining on the most recent 3 months of data (including the new 'electronics' category) will adapt the model to the current data distribution, improving accuracy with minimal downtime by deploying via a new Vertex AI endpoint. Option A (switching to a more complex model without retraining) would not address the drift and may overfit. Option B (routing predictions to human reviewers) is costly, not scalable, and introduces latency. Option D (adjusting prediction thresholds) is insufficient because the model's internal representations are outdated due to the distribution shift, and thresholds alone cannot correct inaccurate predictions.

Answer

Increase the complexity of the model by switching from a feedforward neural network to a gradient boosted tree ensemble, and then deploy without retraining.

Answer

Route all predictions to human reviewers until the model can be re-evaluated, and then manually correct the outputs.

Answer

Adjust the prediction threshold for the 'product_price' feature to account for the price shift, and monitor for another month.

Question 12

Which TWO options are best practices for reducing model serving latency on Vertex AI Endpoints? (Choose two.)

Accepted Answer

Optimize the model using quantization or pruning. Options B and C are correct. Optimizing the model using quantization or pruning reduces the model's size and computational requirements, directly decreasing per-request latency. Deploying the model in the same region as the clients minimizes network round-trip time, reducing overall serving latency. Option A (larger machine type) may increase throughput but does not necessarily reduce latency per request; Option D (batch prediction) is designed for high throughput, not low latency; Option E (model caching) is not a standard feature of Vertex AI endpoints for reducing latency.

Answer

Use a larger machine type with more memory

Answer

Use batch prediction instead of online prediction

Answer

Enable model caching at the endpoint

Question 13

A company runs batch predictions on a large dataset using Vertex AI Batch Prediction. They want to reduce costs without significantly increasing processing time. Which three actions should they take? (Choose three.)

Accepted Answer

Use preemptible VMs for the batch prediction job.. Options A, C, and E are correct. A uses preemptible VMs which are cheaper. C uses custom machine types to avoid overprovisioning and reduce costs. E tunes batch size to maximize throughput per worker, reducing the number of workers needed. Option B increases machine size, which may increase cost per worker. Option D uses TPUs, which are more expensive and may not be beneficial for all model types.

Answer

Use a larger machine type to reduce the number of workers.

Answer

Use TPUs instead of GPUs to accelerate processing.

Question 14

Which TWO are best practices for deploying models to Vertex AI Prediction? (Choose 2.)

Accepted Answer

Monitor prediction latency and error rates with Cloud Monitoring alerts.. The correct answers are A and C. Option A is a best practice because monitoring prediction latency and error rates with Cloud Monitoring alerts helps detect performance issues and ensure availability. Option C is a best practice because using a dedicated service account with minimal permissions follows the principle of least privilege, enhancing security. Option B is not a best practice because logging all raw inputs and outputs for every request can cause privacy concerns, increase costs, and is unnecessary for most use cases. Option D is not a best practice because deploying in the same environment as training may not always be feasible; instead, models should be containerized to ensure consistency across environments. Option E is not a best practice because using the default alias 'default' for all deployments can lead to confusion and makes rollbacks more difficult; proper versioning and aliases should be used.

Answer

Log all raw prediction inputs and outputs for every request for auditing.

Answer

Always deploy the model in the same environment as training to avoid incompatibility.

Answer

Use the default model version alias 'default' for all deployments to simplify updates.

Question 15

A company deploys a model on Vertex AI Endpoints for real-time inference. They notice latency spikes during peak hours. Which action is most effective to reduce latency without sacrificing accuracy?

Accepted Answer

Enable autoscaling based on CPU utilization. Enabling autoscaling based on CPU utilization dynamically adjusts the number of instances to handle traffic spikes, reducing latency without sacrificing accuracy. Option B increases cost without addressing scaling elasticity. Option C may reduce accuracy. Option D does not address latency spikes at the serving layer.

Answer

Use a larger machine type

Answer

Reduce model size by pruning

Answer

Implement client-side caching

Question 16

A company needs to maintain an audit trail of model changes for compliance. Multiple teams will be updating models. What is the best approach to track who created, modified, or deployed each model version?

Accepted Answer

Use Vertex AI Experiments and Metadata to track model lineage and audit logs. Vertex AI Experiments and Metadata provide a built-in way to track model lineage, including who created, modified, or deployed each version, with automatic audit logging. Option A is wrong because Cloud Storage audit logs only capture access to storage objects, not model-specific metadata or changes. Option B is wrong because Cloud Logging requires manual correlation of log entries to model versions, which is error-prone and not designed for lineage tracking. Option D is wrong because a shared spreadsheet lacks automation, access control, and auditability, making it unsuitable for compliance.

Answer

Enable Cloud Storage audit logs and require all model files to be stored in a bucket

Answer

Use Cloud Logging to collect logs from all services and search for model names

Answer

Ask team members to maintain a shared spreadsheet of changes

Question 17

A team is scaling their prototype inference model to handle high-throughput requests with low latency. They use a custom container on Vertex AI Prediction. They notice that latency spikes occur under heavy load. What is the most effective strategy?

Accepted Answer

Optimize model serving with batching and model warm-up.. Optimizing model serving with batching combines multiple requests into a single inference call, improving throughput, and model warm-up pre-loads models into memory, reducing cold-start latency spikes. Option A is wrong because auto-scaling with higher minimum replicas helps with availability but does not address per-request latency spikes; it can add replicas only after load increases. Option C is wrong because adding more CPUs may not reduce latency if the bottleneck is model inference computation or memory bandwidth; batching and warm-up are more targeted. Option D is wrong because switching to GPUs can improve throughput for compute-heavy models but may not specifically reduce latency spikes caused by request bursts; batching and warm-up directly mitigate transient spikes.

Answer

Enable auto-scaling with a higher minimum number of replicas.

Answer

Use a larger machine type with more CPUs.

Answer

Use a GPU-based machine.

Question 18

A machine learning engineer wants to manage multiple model versions and facilitate collaboration across teams. The goal is to track model lineage, versioning, and approvals. Which Vertex AI service should they use?

Accepted Answer

Vertex AI Model Registry. Vertex AI Model Registry provides model versioning, approval tracking, and lineage tracking, enabling easy collaboration across teams. Option B is wrong because ML Metadata is a lower-level tool for tracking metadata, not a user-friendly service for model versioning and approvals. Option C is wrong because Feature Store is designed for managing feature data, not models. Option D is wrong because Vizier is used for hyperparameter tuning, not model management.

Answer

Vertex AI ML Metadata

Answer

Vertex AI Feature Store

Answer

Vertex AI Vizier

Question 19

Refer to the exhibit. A team deploys a model using Cloud Run. They notice that after scaling up, the new instances take about 90 seconds to become ready and serve requests. They want to reduce this startup time. Which configuration change is most likely to help?

Accepted Answer

Change the container image to use a smaller base image. Using a smaller container image (e.g., a minimal base image) reduces image pull and initialization time, directly lowering startup latency. Option A reduces the startupProbe initialDelaySeconds, but this only shortens the time before the probe starts checking; the instance still needs to become ready. Option C reduces memory limit, which could cause out-of-memory errors if the model requires more memory, and does not improve startup time. Option D increases containerConcurrency, which affects how many requests each instance handles but does not affect startup time.

Answer

Reduce the startupProbe initialDelaySeconds to 30

Answer

Reduce the memory limit to 4Gi

Answer

Increase the containerConcurrency to 100

Question 20

You are using Vertex AI Prediction with a custom container that requires a large model file (5 GB). Deployment takes 10 minutes to start. You want to reduce cold start latency. Which action would be MOST effective?

Accepted Answer

Set minReplicas to 1 to keep at least one instance always running.. Cold start latency occurs when a new instance is started and must load the 5 GB model from disk, which can take 10 minutes. Setting minReplicas to 1 ensures that at least one instance is always running and serving, so the model is already loaded in memory, completely avoiding cold start on prediction requests. While other options (compressing the model, using local SSD, or switching to batch prediction) may reduce load time or avoid online serving, they do not eliminate the cold start penalty as effectively as maintaining a warm replica.

Answer

Compress the model file and decompress on startup.

Answer

Use a machine type with local SSD to speed up model loading.

Answer

Switch to batch prediction to avoid online cold start.