PMLE Scaling prototypes into ML models Practice Test 3 — 15 Questions

Question 1

An ML engineer needs to monitor a deployed model for data drift. They want to compare the distribution of incoming predictions against a baseline distribution. Which Vertex AI service should they use?

Accepted Answer

Vertex AI Model Monitoring. Vertex AI Model Monitoring is the correct service because it is specifically designed to detect data drift and feature skew in deployed models. It continuously compares the distribution of incoming prediction requests against a baseline distribution (e.g., training data or a previous window) and alerts the engineer when statistically significant drift is detected, using metrics like Jensen-Shannon divergence or L-infinity distance.

Answer

Vertex AI Feature Store

Answer

Vertex AI Experiments

Answer

Vertex AI Explainable AI

Question 2

A team uses Vertex AI Feature Store to serve features for online predictions. They notice that the online serving latency is high for certain features. The features are stored in a BigQuery source with high cardinality. What is the best practice to reduce latency?

Accepted Answer

Use feature store caching with a larger cache size.. Option B is correct because caching can reduce repeated access to BigQuery. Option A might help but not directly address high cardinality; option C would not integrate with Feature Store; option D is a workaround but not best practice.

Answer

Use batch prediction instead of online prediction.

Answer

Move the features to Cloud Storage and read them directly.

Answer

Increase the number of nodes in the feature store cluster.

Question 3

A large e-commerce company deploys a recommendation model on Vertex AI with autoscaling enabled. During Black Friday, traffic spikes rapidly. The autoscaler adds new instances, but new instances take several minutes to become ready (cold start). As a result, many requests time out. What should they do to mitigate this issue?

Accepted Answer

Set a higher minimum number of instances to handle the expected peak.. Option D is correct because setting a higher minimum number of instances ensures that a baseline capacity is always running and ready to serve traffic. This pre-warms instances, eliminating the cold-start latency during rapid traffic spikes, such as Black Friday, because new instances do not need to initialize from scratch.

Answer

Use a larger machine type to reduce the number of instances needed.

Answer

Configure the autoscaler to use CPU utilization metric instead of request count.

Answer

Increase the health check grace period for new instances.

Question 4

A machine learning engineer is exporting a trained model from Vertex AI Training to the Model Registry. Which artifact should they upload as the model artifact?

Accepted Answer

The saved model directory containing the model file(s) and any custom dependencies.. When exporting a trained model from Vertex AI Training to the Model Registry, the correct artifact is the saved model directory that contains the model file(s) (e.g., SavedModel format for TensorFlow, model.pkl for scikit-learn) along with any custom dependencies required for serving. This ensures the model can be deployed consistently to endpoints or batch predictions, as the Model Registry expects a self-contained artifact that includes both the model binary and its runtime dependencies.

Answer

Only the model checkpoint file (.ckpt or .h5).

Answer

The entire training directory including training code and logs.

Answer

A zip file of the training source code.

Question 5

A company has a TensorFlow model that uses custom operations compiled as .so files. They want to deploy it on Vertex AI for online predictions. The model runs correctly when loaded locally. However, on Vertex AI, the prediction fails with a 'Op type not registered' error. What is the most likely reason?

Accepted Answer

The custom ops were compiled for a different CPU architecture.. Option D is correct because custom TensorFlow operations compiled as .so files are architecture-specific. If the local machine uses a different CPU architecture (e.g., x86_64 with AVX2) than the Vertex AI serving nodes (e.g., x86_64 without AVX2 or ARM), the dynamic library will fail to load, causing the 'Op type not registered' error. The model runs locally because the ops are available, but on Vertex AI the shared object cannot be loaded, so TensorFlow cannot register the custom kernels.

Answer

The model is using a deprecated TensorFlow version.

Answer

The custom ops are not included in the model directory.

Answer

The prediction request format is incorrect.

Question 6

An organization runs a batch prediction job on Vertex AI for a large dataset (10 TB). The job is configured to use a cluster of 100 n1-standard-16 machines. Midway through, the job fails with 'Out of memory' errors. What is the most effective mitigation strategy?

Accepted Answer

Use a machine type with more memory per instance.. The 'Out of memory' error indicates that individual worker nodes are running out of RAM when processing their assigned data shards. Using a machine type with more memory per instance (e.g., n1-highmem-16) directly addresses the root cause by providing each node with sufficient memory to hold the model and its intermediate computations, without changing the data distribution or parallelism strategy.

Answer

Split the input data into smaller chunks and run multiple jobs.

Answer

Enable model parallelism within the prediction script.

Answer

Increase the number of machines to distribute data more.

Question 7

An ML team is deploying a model to Vertex AI for the first time. Which THREE are best practices for scaling from prototype to production?

Accepted Answer

Store all features in a Feature Store for consistency.. Storing all features in a Feature Store (Option B) ensures consistency between training and serving, preventing training-serving skew. Vertex AI Feature Store provides a centralized repository for feature values, enabling reuse, point-in-time lookups, and online serving with low latency, which is critical for production reliability.

Answer

Manually scale instances based on historical traffic patterns.

Answer

Use a single large instance to simplify management.

Question 8

A company has a TensorFlow model that requires GPU for inference. They are deploying on Vertex AI. Which TWO configurations are necessary to ensure GPU is used?

Accepted Answer

Select a machine type that includes a GPU (e.g., NVIDIA Tesla T4).. Option D is correct because Vertex AI requires you to explicitly select a machine type that includes a GPU (e.g., n1-standard-4 with an attached NVIDIA Tesla T4) to provide the physical hardware for GPU acceleration. Without selecting a GPU machine type, the inference will run on CPU only, regardless of any other configuration.

Answer

Set the environment variable TF_GPU_ALLOCATOR=cuda_malloc_async.

Answer

Use the pre-built TensorFlow serving container, which automatically uses GPU if available.

Answer

Build a custom container with GPU drivers.

Question 9

A team is troubleshooting a Vertex AI Pipelines run that keeps failing at the model evaluation step. The pipeline includes steps: data preprocessing, training, evaluation, and deployment. Which THREE actions should they take to diagnose the issue?

Accepted Answer

Verify that the training step output is correctly linked as input to evaluation.. Option A is correct because Vertex AI Pipelines relies on precise input/output artifact linking between steps. If the training step's output (e.g., a model artifact or evaluation metrics) is not correctly wired as the input to the evaluation step, the pipeline will fail due to missing or mismatched data. This is a common misconfiguration in Kubeflow Pipelines DSL, where step outputs must be explicitly passed as arguments to downstream components.

Answer

Increase the memory of the evaluation step's machine.

Answer

Replace the evaluation step with a Vertex AI Model Evaluation service.

Question 10

An ML engineer runs this command to upload a model. The model artifact in Cloud Storage is a directory containing model.pkl and a custom preprocessing script. What will happen when he later deploys this model to an endpoint and sends a prediction request?

Accepted Answer

The prediction will fail because the custom preprocessing script is not a standard scikit-learn serialized object.. Option C is correct because the pre-built container for scikit-learn expects a single serialized model file (e.g., model.pkl) as the artifact. A directory containing a custom preprocessing script is not a standard scikit-learn serialized object, so the container cannot load or execute it, causing the prediction to fail.

Answer

The prediction will succeed because the pre-built container automatically detects and uses the custom preprocessing script.

Answer

The prediction will succeed only if he also specifies a custom prediction routine.

Answer

The prediction will fail because the artifact URI must point to a single file not a directory.

Question 11

A team has deployed a model with autoscaling configured as shown. They notice that during off-peak hours, the endpoint consistently runs 3 instances instead of scaling down to 1. What is the most likely cause?

Accepted Answer

There is a sustained request rate that prevents scaling down.. The autoscaling configuration is likely based on a target metric (e.g., requests per second or CPU utilization). During off-peak hours, if there is a sustained but low request rate that still exceeds the scale-down threshold, the model will not reduce instances below the number needed to handle that load. The endpoint runs 3 instances because the sustained request rate prevents the scaling-down logic from triggering, even though the traffic is lower than peak.

Answer

The `enableAccessLogging` flag increases resource usage.

Answer

The `minReplicaCount` is set too high.

Answer

The model is too large to fit on a single instance.

Question 12

A team trains a distributed TensorFlow model using the config above. After training, they deploy the model for online predictions. The model returns poor quality predictions. They suspect that the model was not trained correctly due to a configuration error. What is the most likely mistake?

Accepted Answer

The `scaleTier` is set to 'STANDARD_1' which only supports up to 3 workers.. Option B is correct because 'STANDARD_1' scale tier is for small scale, max workers is 3. The config set 10 workers, which would be ignored or cause error. The training might have run with fewer workers, leading to poor model. Option A: not required; option C: model-dir is fine; option D: not indicated.

Answer

The training job is using a custom container that does not match the requirements.

Answer

The model was exported incorrectly because the training job did not specify a `--model-export-path`.

Answer

The parameter server count should be at least equal to the worker count.

Question 13

A team has developed a prototype of a recommendation model using a small dataset on a single VM. They need to scale to a larger dataset for production training. They plan to use Vertex AI training with a custom container. What is the best practice for handling the increased data volume?

Accepted Answer

Use TFRecord format and streaming reads.. Option B is correct because using TFRecord format with streaming reads allows efficient, scalable data loading from Cloud Storage, reducing memory pressure and improving I/O performance. Option A is wrong because storing all data in memory is not scalable. Option C is wrong because increasing batch size to maximum can cause memory issues and may not improve throughput. Option D is wrong because a single powerful VM still has limits and is not cost-effective for large datasets.

Answer

Increase the batch size to maximum.

Answer

Store all data in memory before training.

Answer

Use a single powerful VM with high memory.

Question 14

An ML team is moving from a prototype Jupyter notebook to a production training pipeline. They want to ensure reproducibility. Which approach should they take?

Accepted Answer

Use a container with fixed dependencies and record hyperparameters.. Option C is correct because using a container with fixed dependencies and recording hyperparameters ensures that the training environment and configuration are captured, enabling exact reproduction. Option A is wrong because a .py file does not capture the full environment. Option B is wrong because exporting the notebook's output model directly lacks environment tracking. Option D is wrong because interactive tuning is not reproducible.

Answer

Use interactive parameter tuning.

Answer

Export the notebook's output model directly.

Answer

Save the notebook as a .py file.

Question 15

A data scientist trained a model on a single GPU but needs to train on multiple GPUs for a larger dataset. They observe that training time does not decrease linearly with additional GPUs. Which common issue is most likely?

Accepted Answer

Data pipeline bottleneck.. Option A is correct because a data pipeline bottleneck can starve GPUs, preventing linear speedup. Option B is wrong because overfitting relates to model performance, not training speed. Option C is wrong because learning rate affects convergence, not parallelism efficiency. Option D is wrong because model architecture size does not directly cause non-linear speedup.

Answer

Overfitting.

Answer

Model architecture too simple.

Answer

Learning rate too high.