PMLE Scaling Prototypes into ML Models Practice Test 4 — 15 Questions

Question 1

You are developing a custom container to train a PyTorch model on Vertex AI. The container must read training data from a Cloud Storage bucket and write checkpoints to the same bucket. After building the container locally, you push it to Artifact Registry. When you submit a Vertex AI custom training job, the job fails immediately with a permission error when accessing Cloud Storage. The service account attached to the training job has the Storage Object Admin role. What is the most likely cause of the failure?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The container's entrypoint script does not use the google-auth library to obtain credentials from the metadata server, so it cannot authenticate to Cloud Storage.. The correct answer is that the container must use a library like google-auth to obtain credentials from the metadata server. Vertex AI injects a service account token into the container, but the application code must retrieve and use it. Without this, authentication fails, causing permission errors. The service account's IAM roles are necessary but not sufficient; the container must also correctly implement authentication.

Answer

The container image does not have the necessary permissions to access the service account token because it runs as a non-root user without the required Linux capabilities.

Answer

The Vertex AI training service uses the default Compute Engine service account instead of the custom service account specified in the job configuration.

Answer

The container does not include the Cloud Storage client library, so it cannot authenticate.

Question 2

An ML engineer is using Vertex AI Pipelines to orchestrate a training workflow. The pipeline includes a component that trains a model and another that evaluates it. The evaluation component must only run if the training component succeeds, and the pipeline should fail immediately if training fails. Which Kubeflow Pipelines control flow construct should the engineer use?

Accepted Answer

Rely on the default behavior: components run in order, and if the training component fails, the pipeline fails and the evaluation component does not run.. Vertex AI Pipelines uses the default failure semantics of Kubeflow Pipelines: when a component fails, the pipeline fails and downstream components are not executed. By defining the evaluation component to depend on the training component, the engineer ensures it only runs after training succeeds. The pipeline fails immediately on training failure without additional constructs.

Answer

Use a condition that checks the training component's output status and skips the evaluation component if training fails.

Answer

Use an ExitHandler to catch the training component failure and then conditionally run the evaluation component.

Answer

Use the after() method to specify that the evaluation component runs after the training component.

Question 3

You are deploying a trained model to Vertex AI for online prediction. The model requires a GPU for low-latency inference. You need to configure the endpoint to automatically scale based on traffic while keeping costs under control. Which scaling configuration should you use?

Accepted Answer

Set a minimum replica count of 0 and a maximum replica count of 10, with autoscaling based on GPU utilization.. Autoscaling on GPU utilization ensures that the endpoint scales based on actual GPU load, which is the bottleneck for GPU inference. Setting a minimum of 0 replicas allows scaling to zero during idle periods, reducing costs. This configuration is ideal for cost-sensitive deployments with variable traffic.

Answer

Set a minimum replica count of 1 and a maximum replica count of 10, with autoscaling based on CPU utilization.

Answer

Set a minimum replica count of 1 and a maximum replica count of 10, with autoscaling based on GPU utilization.

Answer

Set a minimum replica count of 0 and a maximum replica count of 10, with autoscaling based on request count per second.

Question 4

You are training a TensorFlow model on Vertex AI using a custom training job with a single node and multiple GPUs. You want to use the MirroredStrategy for distributed training. The training data is stored in Cloud Storage as TFRecord files. You need to ensure that the training job scales efficiently across the GPUs. Which two actions should you take? (Choose two.)

Accepted Answer

Use a tf.data.Dataset pipeline that reads from Cloud Storage and applies prefetching and parallel interleave.. To efficiently scale training across multiple GPUs with MirroredStrategy, you must ensure the input pipeline can feed data fast enough and that the batch size is appropriate. Using tf.data with parallel interleave and prefetching keeps the GPUs fed, while a batch size divisible by the number of GPUs ensures balanced workload. These two actions directly address data throughput and compute distribution.

Answer

Configure the training job to use a TPU instead of GPUs to take advantage of XLA compilation.

Answer

Set the strategy to MirroredStrategy with cross_device_ops set to HierarchicalCopyAllReduce.

Answer

Use a tf.distribute.MirroredStrategy with a single GPU and rely on automatic model parallelism.

Question 5

An ML engineer is preparing a Vertex AI custom training job that uses a distributed TensorFlow model across multiple nodes. The job must be resilient to worker restarts and must allow training to resume without starting over. The code uses tf.distribute.MultiWorkerMirroredStrategy. Which two actions should the engineer take? (Choose two.)

Accepted Answer

Configure the training application to save checkpoints to a Cloud Storage location using tf.keras.callbacks.ModelCheckpoint or tf.train.CheckpointManager.. Resilient distributed training with MultiWorkerMirroredStrategy requires durable checkpointing and a shared location all workers can access. Saving checkpoints to Cloud Storage preserves model and optimizer state, and using a shared path ensures every worker can read the latest checkpoint after a restart. TensorBoard logging, preemptible VMs, and epoch or batch size changes do not provide the resumption capability the scenario requires.

Answer

Increase the number of epochs and batch size so the job finishes before any potential restart.

Answer

Set the worker pool to use preemptible VMs to reduce cost and rely on Vertex AI to restart the job automatically.

Answer

Enable Vertex AI TensorBoard integration and log metrics to monitor training progress.

Question 6

You are scaling a prototype ML model to production on Vertex AI. The model is a PyTorch model that you trained locally. You need to deploy it to a Vertex AI Endpoint for online predictions. You want to minimize latency and ensure the model can handle high throughput. You also need to be able to roll back to a previous version quickly if issues arise. What should you do?

Accepted Answer

Deploy the model to a Vertex AI Endpoint using a custom container that serves the PyTorch model, and use Vertex AI's traffic splitting to direct a small percentage of traffic to the new version for canary testing, while keeping the previous version deployed.. For PyTorch models, a custom container is required because no pre-built serving container exists. Deploying multiple versions to an endpoint and using traffic splitting enables canary testing and instant rollback by shifting traffic. This approach ensures high availability, low latency, and safe updates.

Answer

Deploy the model to a Vertex AI Endpoint using a pre-built PyTorch container, and set the endpoint to automatically scale based on CPU utilization to handle high throughput.

Answer

Deploy the model to a Vertex AI Endpoint using a custom container, and configure the endpoint to use a GPU for each replica to reduce latency. Use Vertex AI's built-in A/B testing feature to compare model versions.

Answer

Deploy the model to a Vertex AI Endpoint with a single replica to minimize latency, and use Vertex AI Model Registry to manage versions and roll back by redeploying the previous model version.

Question 7

You are deploying a scikit-learn model to a Vertex AI Endpoint for online serving. The model was trained on a large dataset and has a high memory footprint. You want to minimize inference latency while ensuring the model stays loaded in memory. Which approach should you take?

Accepted Answer

Deploy the model to a Vertex AI Endpoint with a machine type that has sufficient memory and set the minReplicaCount to 1, and ensure the model is loaded during deployment by setting the model's container to load the model at startup.. To minimize inference latency and keep the model loaded, you need to ensure the model is loaded at container startup and that at least one replica remains running. Setting minReplicaCount to 1 prevents scale-to-zero, and a machine type with sufficient memory avoids out-of-memory errors. Loading the model at startup avoids on-demand loading during the first request, which would increase latency.

Answer

Deploy the model to a Vertex AI Endpoint with autoscaling enabled and set the minReplicaCount to 1.

Answer

Deploy the model to a Vertex AI Endpoint with a single large machine type and set the minReplicaCount to 1.

Answer

Deploy the model to a Vertex AI Endpoint with a machine type that has sufficient memory and set the minReplicaCount to 1, and enable request-response logging.

Question 8

You have a prototype model packaged as a scikit-learn pipeline saved with joblib. You need to deploy it to Vertex AI for online prediction with low latency and the ability to scale to zero when there is no traffic. The model must receive JSON requests and return class probabilities. What should you do?

Accepted Answer

Write a custom container that loads the joblib pipeline and implements the Vertex AI HTTP prediction contract, push it to Artifact Registry, upload the model with that container, deploy to an endpoint, and configure autoscaling.. Serving a scikit-learn pipeline that must accept JSON and emit class probabilities requires control over request parsing and response formatting. A custom container that loads the joblib artifact and implements the Vertex AI HTTP prediction contract provides that control. Uploading the model with that container and deploying it to an endpoint with autoscaling yields a scalable, low-latency online prediction service.

Answer

Deploy the joblib file to a Vertex AI endpoint using the pre-built XGBoost container, since it can load generic Python objects and expose a predict method.

Answer

Upload the joblib file as a Vertex AI Model with a pre-built scikit-learn prediction container, deploy it to an endpoint with a custom-trained model server, and enable autoscaling with minReplicaCount set to 0.

Answer

Convert the scikit-learn pipeline to a TensorFlow SavedModel using tf.keras, upload it to Vertex AI Model Registry, and deploy it with a pre-built TensorFlow Serving container.

Question 9

A team is deploying a TensorFlow model to Vertex AI for online predictions. The model must process incoming requests with a custom preprocessing function that includes a lookup table. The lookup table is updated weekly. The team wants to minimize downtime and avoid rebuilding the serving container each time the lookup table changes. What should they do?

Accepted Answer

Use a custom serving container that loads the lookup table from a Cloud Storage bucket into memory at startup and periodically checks for updates using a background thread.. The best solution is to decouple the lookup table from the container image and load it dynamically. A custom container can download the table at startup and then periodically check for updates in a background thread. This avoids rebuilding the image and restarting the service, thus minimizing downtime. It also allows the team to update the table by simply uploading a new version to Cloud Storage.

Answer

Store the lookup table in a Cloud Storage bucket and have the custom serving container download it at startup, then restart the container weekly.

Answer

Mount a Cloud Storage bucket as a volume in the serving container and read the lookup table from the mounted path on each request.

Answer

Bake the lookup table into the container image and redeploy the model each week with a new image version.

Question 10

An ML engineer is using Vertex AI Training to fine-tune a large language model (LLM) using a custom container. The training job runs on a single node with 8 NVIDIA A100 GPUs. The engineer wants to reduce training time by using data parallelism with PyTorch DistributedDataParallel (DDP). Which two steps are necessary to correctly configure the training job? (Choose two.)

Accepted Answer

Use `torch.nn.parallel.DistributedDataParallel` to wrap the model, and launch one process per GPU.. To use PyTorch DDP on a single node with multiple GPUs, you must set up the distributed environment by specifying the master address and port for process group initialization. Additionally, you need to wrap the model in DistributedDataParallel and launch one process per GPU to achieve data parallelism. These steps ensure that gradients are synchronized correctly across GPUs, reducing training time.

Answer

Configure the training job to use a TPU instead of GPUs, as DDP is not supported on GPUs.

Answer

Set the `NCCL_DEBUG` environment variable to `INFO` to enable NCCL communication.

Answer

Ensure that the custom container includes the `torch.distributed` package and that the training script calls `torch.distributed.init_process_group`.

Question 11

You have a TensorFlow training script that runs on a single machine. To speed up training on Vertex AI with 8 GPUs on a single machine, which strategy should you use?

Accepted Answer

tf.distribute.MirroredStrategy. tf.distribute.MirroredStrategy is designed for synchronous, data-parallel training across multiple GPUs on a single machine. It replicates the model on each GPU, splits each batch across replicas, and uses all-reduce (via NCCL) to aggregate gradients, which is exactly the scenario described: 8 GPUs on one machine. This is the canonical strategy for single-node multi-GPU TensorFlow training.

Answer

tf.distribute.ParameterServerStrategy

Answer

tf.distribute.TPUStrategy

Answer

tf.distribute.MultiWorkerMirroredStrategy

Question 12

A data science team is building a feature engineering pipeline that processes large-scale data from BigQuery daily. They need to compute aggregate features and store the results in Vertex AI Feature Store for both online serving and offline training. Which Google Cloud service is best suited for this batch computation?

Accepted Answer

Dataflow. Dataflow is Google Cloud's managed Apache Beam service, purpose-built for batch and streaming data pipelines that read from BigQuery, transform data, and write to sinks like Vertex AI Feature Store. It handles autoscaling, sharding, and windowing natively, making it the canonical choice for daily batch feature engineering at scale. Its native BigQuery and Feature Store I/O connectors mean minimal glue code.

Answer

Cloud Composer

Answer

Dataproc

Answer

Cloud Functions

Question 13

You are fine-tuning a large language model (LLM) from Hugging Face Transformers using Vertex AI Training. The model has 7 billion parameters and does not fit into the memory of a single GPU. You need to train across multiple GPUs, splitting the model layers across devices. Which distributed training approach should you use?

Accepted Answer

Model parallelism using pipeline parallelism. When a model is too large to fit on a single GPU, model parallelism is required to split the model's layers across multiple devices. Pipeline parallelism is a specific form of model parallelism that partitions layers into stages and pipelines micro-batches across devices, enabling training of models like a 7B-parameter LLM across multiple GPUs. This is the correct approach when memory, not throughput, is the binding constraint.

Answer

Data parallelism with MultiWorkerMirroredStrategy

Answer

Mixed precision training (FP16)

Answer

Data parallelism with tf.distribute.MirroredStrategy

Question 14

A company is using Vertex AI Vizier for hyperparameter tuning of a model with 5 integer hyperparameters, each with a range of 10-100. They have a budget of 50 trials and want to maximize the chance of finding the best configuration. Which Vizier algorithm should they use?

Accepted Answer

Bayesian optimization (GP bandit). Bayesian optimization (GP bandit) is Vizier's default and most sample-efficient algorithm, using a Gaussian Process surrogate model to balance exploration and exploitation across the 5-dimensional hyperparameter space. With only 50 trials over a large search space, it converges on promising regions far faster than uninformed methods. This makes it the best choice for maximizing the chance of finding the optimal configuration within a limited budget.

Answer

Grid search

Answer

Simulated annealing

Answer

Random search

Question 15

You want to use a pre-trained model from TensorFlow Hub for image classification, but you need to adapt it to classify your own custom categories with a small dataset. Which Vertex AI approach is most appropriate?

Accepted Answer

Write a custom training script that loads the pre-trained model and fine-tunes it on your dataset. Fine-tuning a pre-trained model on your custom dataset is the most appropriate approach when you have a small dataset and need to adapt the model to new categories. This leverages transfer learning, where the pre-trained weights are used as a starting point and updated with your data.

Answer

Deploy the pre-trained model as-is via Vertex AI JumpStart

Answer

Build a custom container with the pre-trained model and deploy to Vertex AI Endpoints

Answer

Use Vertex AI AutoML for image classification