PMLE Scaling Prototypes into ML Models — 20 Questions

Question 1

You want to deploy a trained scikit-learn model to Vertex AI for online predictions. The model file is 2 GB. Which option should you use?

Accepted Answer

Use a custom container with the scikit-learn model. Vertex AI's pre-built scikit-learn container has a model size limit of 500 MB. Since the model is 2 GB, it exceeds this limit; therefore, you must use a custom container that allows larger model sizes and provides more control over dependencies and runtime.

Answer

Use AI Platform (now Vertex AI) with a large model VM

Answer

Use Cloud Functions to host the model

Answer

Upload the model to Vertex AI Model Registry and deploy using pre-built scikit-learn container

Question 2

A company wants to bring their own Docker container to Vertex AI for training a model with a custom framework. They need to ensure the container is compatible with the Vertex AI training service. What is the minimum requirement for the container?

Accepted Answer

The container must include a Python script that reads command-line arguments and outputs a trained model to /tmp/model. Vertex AI custom container training expects the container to run a training script that reads command-line arguments (e.g., hyperparameters and data paths) and outputs the trained model. An HTTP server on port 8080 is not required; the training service launches the container and passes arguments via command-line. Health checks are handled internally by Vertex AI, not by the container itself.

Answer

The container must be based on a Google-provided base image

Answer

The container must expose an HTTP server on port 8080 that responds to health checks and starts the training job when called

Answer

The container must expose an HTTP server on port 8080 and handle /health and /predict endpoints

Question 3

A team is building a feature pipeline for an ML model. They need to compute aggregate features over a sliding time window from streaming data. Which Google Cloud service is most appropriate for this task?

Accepted Answer

Cloud Dataflow with fixed windows. Cloud Dataflow is the most appropriate service because it natively supports sliding time windows via the Apache Beam model. While the option mentions 'fixed windows', Cloud Dataflow can also implement sliding windows, which are required for computing aggregate features over overlapping intervals. The other services lack native sliding window aggregation capabilities.

Answer

Cloud Pub/Sub for windowing logic

Answer

BigQuery scheduled queries

Answer

Cloud Functions with Pub/Sub triggers

Question 4

You have a TensorFlow training script that runs on a single machine. To speed up training on Vertex AI with 8 GPUs on a single machine, which strategy should you use?

Accepted Answer

tf.distribute.MirroredStrategy. MirroredStrategy performs synchronous data parallelism across multiple GPUs on a single machine. MultiWorkerMirroredStrategy is for multiple machines, not needed here. ParameterServerStrategy is for distributed asynchronous training. TPUStrategy is for TPUs.

Answer

tf.distribute.ParameterServerStrategy

Answer

tf.distribute.TPUStrategy

Answer

tf.distribute.MultiWorkerMirroredStrategy

Question 5

A data science team is building a feature engineering pipeline that processes large-scale data from BigQuery daily. They need to compute aggregate features and store the results in Vertex AI Feature Store for both online serving and offline training. Which Google Cloud service is best suited for this batch computation?

Accepted Answer

Dataflow. Dataflow is ideal for batch processing large datasets from BigQuery with Apache Beam. It can write directly to Feature Store's API. Cloud Functions is event-driven and not for heavy batch. Dataproc is for Spark/Hadoop, not as efficient for Beam. Cloud Composer is an orchestrator, not executor.

Answer

Cloud Composer

Answer

Dataproc

Answer

Cloud Functions

Question 6

You are fine-tuning a large language model (LLM) from Hugging Face Transformers using Vertex AI Training. The model has 7 billion parameters and does not fit into the memory of a single GPU. You need to train across multiple GPUs, splitting the model layers across devices. Which distributed training approach should you use?

Accepted Answer

Model parallelism using pipeline parallelism. Model parallelism (pipeline parallelism) splits model layers across devices, necessary for large models that don't fit on one GPU. Data parallelism replicates the model and splits data, not suitable if model doesn't fit. Mixed precision reduces memory but still requires model parallelism for 7B. Fully sharded data parallelism (FSDP) is a form of data parallelism with sharding, but pipeline parallelism is more common for layer-wise splitting.

Answer

Data parallelism with MultiWorkerMirroredStrategy

Answer

Mixed precision training (FP16)

Answer

Data parallelism with tf.distribute.MirroredStrategy

Question 7

A company is using Vertex AI Vizier for hyperparameter tuning of a model with 5 integer hyperparameters, each with a range of 10-100. They have a budget of 50 trials and want to maximize the chance of finding the best configuration. Which Vizier algorithm should they use?

Accepted Answer

Bayesian optimization (GP bandit). Bayesian optimization (GP bandit) is best for small trial budgets as it uses past results to guide search. Grid search would be too many combinations. Random search is better than grid but still less efficient than Bayesian. Vizier does not support simulated annealing.

Answer

Grid search

Answer

Simulated annealing

Answer

Random search

Question 8

You want to use a pre-trained model from TensorFlow Hub for image classification, but you need to adapt it to classify your own custom categories with a small dataset. Which Vertex AI approach is most appropriate?

Accepted Answer

Write a custom training script that loads the pre-trained model and fine-tunes it on your dataset. Transfer learning fine-tunes a pre-trained model on a new dataset with small data. JumpStart deploys foundation models but not fine-tune for custom categories easily. Custom container is overkill. AutoML requires no code but may not be suitable if you want to control the pre-trained model.

Answer

Deploy the pre-trained model as-is via Vertex AI JumpStart

Answer

Build a custom container with the pre-trained model and deploy to Vertex AI Endpoints

Answer

Use Vertex AI AutoML for image classification

Question 9

Your Vertex AI custom training job is failing with an out-of-memory error on a single GPU. You need to reduce memory usage without changing the model architecture. Which approach should you try first?

Accepted Answer

Decrease the batch size. Decreasing the batch size is the simplest and most direct approach to reduce GPU memory usage for a custom training job. It linearly reduces the memory needed for activations and gradients. While mixed precision can also reduce memory, it may introduce numerical precision issues and only works on compatible hardware, so decreasing batch size should be attempted first.

Answer

Implement model parallelism across GPUs

Answer

Use gradient accumulation

Answer

Enable mixed precision training (FP16)

Question 10

You are deploying a deep learning model on edge devices with limited computational resources. The model must run inference in <10 ms and the model size must be under 50 MB. Currently, your trained model is 200 MB and runs in 50 ms. Which combination of model compression techniques should you apply?

Accepted Answer

Apply post-training quantization (INT8) and pruning. Post-training quantization to INT8 reduces model size by 4x and often speeds up inference. Pruning removes redundant weights, further reducing size. Knowledge distillation would require retraining a smaller student model. Quantization-aware training is more accurate but needs retraining. For a simple fix, quantization and pruning are effective.

Answer

Only apply weight pruning

Answer

Apply quantization-aware training and knowledge distillation

Answer

Use knowledge distillation to train a smaller student model

Question 11

You are running a Vertex AI custom training job with pre-built TensorFlow container. You want to use TPU v3 pods for faster training. Which configuration is required?

Accepted Answer

Specify machine type as tpu-v3-8 in the worker pool spec and use tf.distribute.TPUStrategy. TPU training requires specifying the TPU type in the worker pool spec and using tf.distribute.TPUStrategy in code. The pre-built container supports TPUs. Runtime version must include TPU support.

Answer

Set the worker pool machine type to n1-standard-8 and add --accelerator type=tpu

Answer

Use a custom container with TPU libraries and set --tpu-topology=v3-8

Answer

Enable TPU by setting the environment variable TPU_NAME

Question 12

You need to perform a large-scale feature computation on streaming data from Pub/Sub, transforming raw events into features, and writing results to Vertex AI Feature Store for online serving. Which Google Cloud architecture is most appropriate?

Accepted Answer

Use Dataflow streaming pipeline with Apache Beam to read from Pub/Sub, compute features, and write to Feature Store. Dataflow with streaming (Apache Beam) can read from Pub/Sub, transform data, and write to Feature Store via the online serving API. Cloud Functions is not suitable for complex transforms. Dataproc Streaming is possible but Dataflow is more natural. Cloud Run is for request-response.

Answer

Use Dataproc with Spark Streaming to read from Pub/Sub and write to Feature Store

Answer

Use Cloud Functions triggered by Pub/Sub to compute features and update Feature Store

Answer

Use Cloud Run to consume Pub/Sub messages and update Feature Store via a service

Question 13

You want to use Vertex AI JumpStart to quickly deploy a pre-built foundation model for text summarization. Which action is required?

Accepted Answer

Select the model from Model Garden and deploy it to a Vertex AI endpoint. JumpStart provides one-click deployment of foundation models from Model Garden. You select a model and deploy it to an endpoint. No custom training or container building is needed.

Answer

Train the model from scratch using Vertex AI Training

Answer

Export the model to a Cloud Storage bucket and use batch prediction

Answer

Build a custom Docker container with the model and deploy to Vertex AI

Question 14

Your PyTorch training script uses DistributedDataParallel (DDP) across 4 vertices each with 4 GPUs (16 GPUs total). You submit a Vertex AI custom training job. How should you configure the worker pool spec?

Accepted Answer

Create one worker pool with 4 replicas, each with machine type having 4 GPUs. For DDP across multiple machines, use MultiWorkerMirroredStrategy equivalent in PyTorch: set replicas to 4, each with machine type having 4 GPUs. The TF_CONFIG env var is not needed; Vertex AI sets necessary environment variables for distributed training.

Answer

Create a chief worker pool with 1 replica (4 GPUs) and a parameter server pool with 4 replicas (no GPUs)

Answer

Create 4 separate jobs, each with 1 replica and 4 GPUs

Answer

Create one worker pool with 16 replicas, each with 1 GPU

Question 15

You are fine-tuning a pre-trained BERT model from Hugging Face on a custom text classification dataset using Vertex AI Training. You want to speed up training by using mixed precision. What should you do?

Accepted Answer

Set fp16=True in the TrainingArguments. Hugging Face Trainer supports mixed precision via the fp16 argument. Set it to True in the TrainingArguments. No need to modify the model architecture or use a custom container.

Answer

Modify the model to use half-precision layers

Answer

Use a custom container with TensorFlow instead of PyTorch

Answer

Enable mixed precision via Vertex AI hyperparameter tuning

Question 16

You are designing a distributed training job for a very large neural network that does not fit on a single machine. You need to split the model across multiple devices. Which TWO techniques can you use?

Accepted Answer

Pipeline parallelism. Pipeline parallelism splits layers across devices; model parallelism (operator-level) splits individual operations. Data parallelism replicates the model. ParameterServer is a form of distributed training, not model splitting.

Answer

ParameterServerStrategy

Answer

Data parallelism with MirroredStrategy

Answer

MultiWorkerMirroredStrategy

Question 17

You are fine-tuning a large language model using Vertex AI Training with spot VMs to reduce cost. Your training job keeps getting preempted, causing delays. Which THREE strategies can help mitigate the impact of preemption?

Accepted Answer

Enable checkpointing and restore from the latest checkpoint after preemption. Checkpointing saves progress; using a smaller model reduces training time; increasing number of workers allows faster training and recovery. Using dedicated VMs increases cost. Reducing batch size doesn't help with preemption frequency.

Answer

Reduce batch size to lower memory usage

Answer

Switch to dedicated (non-preemptible) VMs

Question 18

You are building a machine learning pipeline on Google Cloud. You need to perform feature engineering on large datasets stored in BigQuery and store the resulting features in Vertex AI Feature Store for both online and offline use. Which TWO Google Cloud services should you use?

Accepted Answer

Dataflow. Dataflow can read from BigQuery, compute features via Apache Beam, and write to Feature Store. Alternatively, Dataproc can also do this but Dataflow is more serverless. Cloud Functions is not suitable for large-scale. Cloud Build is for CI/CD. BigQuery ML is for in-database ML.

Answer

Cloud Functions

Answer

BigQuery ML

Answer

Cloud Build

Question 19

A data scientist wants to train a TensorFlow model on Vertex AI using a pre-built container. Which of the following pre-built containers is NOT available for custom training in Vertex AI?

Accepted Answer

Apache Spark. Vertex AI provides pre-built containers for TensorFlow, PyTorch, scikit-learn, and XGBoost. There is no pre-built container for Apache Spark in the Vertex AI training service; Spark jobs are typically run on Dataproc.

Answer

TensorFlow

Answer

PyTorch

Answer

scikit-learn

Question 20

You need to run a distributed training job on Vertex AI using TensorFlow with MirroredStrategy on a single machine with 4 GPUs. Which training configuration should you use?

Accepted Answer

Use MirroredStrategy with a single workerPoolSpec containing a machine_type with 4 GPUs. For single-machine multi-worker training with multiple GPUs, TensorFlow's MirroredStrategy is appropriate. The workerPoolSpec should have a single worker pool with a machine type that has multiple GPUs.

Answer

Use MultiWorkerMirroredStrategy with multiple workerPools

Answer

Use MirroredStrategy with two workerPoolSpecs, each with 2 GPUs

Answer

Use ParameterServerStrategy with a chief and a parameter server