Scaling Prototypes into ML Models practice questions

Use AI Platform (now Vertex AI) with a large model VM

Why wrong: Not a specific option; custom container is the correct path.

Use Cloud Functions to host the model

Why wrong: Cloud Functions have memory and timeout limits, not suitable.

Upload the model to Vertex AI Model Registry and deploy using pre-built scikit-learn container

Why wrong: Pre-built container may have model size limits; 2 GB might be too large.

The container must be based on a Google-provided base image

Why wrong: Incorrect. While Google provides base images, they are not mandatory; any compatible Docker image can be used.

The container must expose an HTTP server on port 8080 that responds to health checks and starts the training job when called

Why wrong: Incorrect. Vertex AI does not require an HTTP server on port 8080. The container simply runs the training script via command-line arguments.

The container must expose an HTTP server on port 8080 and handle /health and /predict endpoints

Why wrong: Incorrect. The /predict endpoint is for prediction serving, not training. Also, an HTTP server is not a requirement for training containers.

Cloud Pub/Sub for windowing logic

Why wrong: Cloud Pub/Sub is a messaging service for streaming data ingestion; it does not provide windowing logic or compute features.

BigQuery scheduled queries

Why wrong: BigQuery scheduled queries are batch-oriented and cannot process real-time streaming data with sliding windows.

Cloud Functions with Pub/Sub triggers

Why wrong: Cloud Functions with Pub/Sub triggers can react to individual events but lack native support for stateful sliding-window aggregations.

tf.distribute.ParameterServerStrategy

Why wrong: ParameterServerStrategy is for asynchronous distributed training with parameter servers.

tf.distribute.TPUStrategy

Why wrong: TPUStrategy is for TPU hardware, not GPUs.

tf.distribute.MultiWorkerMirroredStrategy

Why wrong: MultiWorkerMirroredStrategy is for multi-machine distributed training.

Cloud Composer

Why wrong: Cloud Composer is an orchestration tool (Airflow), not a compute engine for feature engineering.

Dataproc

Why wrong: Dataproc can be used but Dataflow is more native for Beam pipelines and has better integration with Feature Store.

Cloud Functions

Why wrong: Cloud Functions is serverless, but limited in memory and execution time, not suitable for large batch feature engineering.

Data parallelism with MultiWorkerMirroredStrategy

Why wrong: Same as A but across machines; still replicates model, not suitable if model doesn't fit per device.

Mixed precision training (FP16)

Why wrong: Mixed precision halves memory but still requires the model to fit on one GPU; 7B may still not fit.

Data parallelism with tf.distribute.MirroredStrategy

Why wrong: Data parallelism replicates the entire model on each GPU, requiring each GPU to hold the model, which is not possible if model doesn't fit.

Grid search

Why wrong: Grid search would require 5^10 combinations, too many for 50 trials.

Simulated annealing

Why wrong: Simulated annealing is not an available algorithm in Vertex AI Vizier.

Random search

Why wrong: Random search selects hyperparameter configurations uniformly at random without using any prior trial outcomes to guide subsequent selections, so it cannot exploit the 50-trial budget to converge efficiently toward the optimal region of the 5-dimensional integer space. It is tempting because it is simple to implement and avoids the overhead of a surrogate model, making it a correct choice when the budget is extremely small relative to the search space and no sequential learning is needed.

Deploy the pre-trained model as-is via Vertex AI JumpStart

Why wrong: JumpStart deploys pre-trained models but they are not tailored to custom categories; you need fine-tuning.

Build a custom container with the pre-trained model and deploy to Vertex AI Endpoints

Why wrong: This deploys the model but does not adapt it to new categories; fine-tuning is needed.

Use Vertex AI AutoML for image classification

Why wrong: AutoML is an option but not the best if you want to leverage a specific pre-trained model from TF Hub.

Implement model parallelism across GPUs

Why wrong: Implementing model parallelism across GPUs is incorrect because the out-of-memory error occurs on a *single* GPU. This approach requires multiple GPUs to distribute the model's layers, which does not reduce the memory footprint on the *initial* failing GPU. It is tempting as it addresses OOM errors for models too large to fit into the memory of *any single GPU*, enabling training by spreading the model's components across several devices.

Use gradient accumulation

Why wrong: Gradient accumulation simulates a larger batch size but does not reduce memory per step; it actually requires storing intermediate gradients and does not resolve the OOM error.

Enable mixed precision training (FP16)

Why wrong: Mixed precision training reduces memory by storing weights and activations in FP16, but the out-of-memory error on a single GPU stems from the model exceeding the GPU’s total VRAM capacity, not from precision overhead. FP16 can even increase memory pressure if the model requires FP32 master weights for gradient updates. It is tempting because mixed precision is commonly used to speed up training and reduce memory on modern GPUs with Tensor Cores, and would be correct if the goal were to accelerate computation or fit a slightly larger batch size on a memory-constrained GPU.

Only apply weight pruning

Why wrong: Pruning alone may not achieve necessary size reduction; quantization is more impactful.

Apply quantization-aware training and knowledge distillation

Why wrong: These require retraining; not the fastest approach if you want to avoid retraining.

Use knowledge distillation to train a smaller student model

Why wrong: Knowledge distillation alone reduces model size by training a smaller student network, but it does not guarantee the student will meet the <10 ms inference latency or the <50 MB size constraint unless the student architecture is explicitly designed for those limits; the stem’s 200 MB, 50 ms model requires additional techniques like pruning or quantisation to achieve both targets. It is tempting because distillation effectively transfers accuracy from a large teacher to a compact student, and would be correct if the only requirement were reducing model size while preserving performance, without strict latency or size thresholds.

Set the worker pool machine type to n1-standard-8 and add --accelerator type=tpu

Why wrong: TPU is not added as accelerator; it's specified via --tpu-topology or machine type.

Use a custom container with TPU libraries and set --tpu-topology=v3-8

Why wrong: Pre-built containers already have TPU support; custom container not needed.

Enable TPU by setting the environment variable TPU_NAME

Why wrong: TPU_NAME is set automatically; the machine type must be specified for Vertex AI.

Use Dataproc with Spark Streaming to read from Pub/Sub and write to Feature Store

Why wrong: Dataproc with Spark Streaming lacks native integration for writing directly to Vertex AI Feature Store’s online serving endpoints; it requires a custom connector or intermediate batch layer, introducing latency that undermines the low-latency online serving requirement. It is tempting because Spark Streaming excels at large-scale, stateful stream processing for batch-oriented feature pipelines, and would be correct if the output target were a data warehouse or filesystem rather than an online feature store.

Use Cloud Functions triggered by Pub/Sub to compute features and update Feature Store

Why wrong: Cloud Functions are short-lived and not designed for complex streaming transformations at scale.

Use Cloud Run to consume Pub/Sub messages and update Feature Store via a service

Why wrong: Cloud Run is stateless and not built for streaming; scaling may cause issues.

Train the model from scratch using Vertex AI Training

Why wrong: JumpStart offers pre-trained models; training from scratch is unnecessary.

Export the model to a Cloud Storage bucket and use batch prediction

Why wrong: JumpStart deploys models for online prediction, not batch.

Build a custom Docker container with the model and deploy to Vertex AI

Why wrong: JumpStart eliminates the need for custom containers.

Create a chief worker pool with 1 replica (4 GPUs) and a parameter server pool with 4 replicas (no GPUs)

Why wrong: DDP does not use parameter servers; this is for TensorFlow with PS strategy.

Create 4 separate jobs, each with 1 replica and 4 GPUs

Why wrong: Separate jobs are not coordinated; DDP requires a single job with multiple workers.

Create one worker pool with 16 replicas, each with 1 GPU

Why wrong: This would create 16 single-GPU nodes, not 4x4.

Modify the model to use half-precision layers

Why wrong: Not necessary; Trainer handles it.

Use a custom container with TensorFlow instead of PyTorch

Why wrong: Mixed precision is available in both frameworks; no need to switch.

Enable mixed precision via Vertex AI hyperparameter tuning

Why wrong: Mixed precision is not a hyperparameter; it's a training configuration.

ParameterServerStrategy

Why wrong: Parameter servers distribute parameters but model is still replicated.

Data parallelism with MirroredStrategy

Why wrong: Data parallelism replicates the model, not splits it.

MultiWorkerMirroredStrategy

Why wrong: This is data parallelism across workers.

Reduce batch size to lower memory usage

Why wrong: Does not prevent preemption; might even increase training time.

Switch to dedicated (non-preemptible) VMs

Why wrong: This increases cost, the opposite of the goal.

Cloud Functions

Why wrong: Not suitable for large batch processing.

BigQuery ML

Why wrong: For creating ML models directly in BigQuery, not for feature engineering pipelines.

Cloud Build

Why wrong: CI/CD tool, not for feature engineering.

TensorFlow

Why wrong: Available pre-built container.

PyTorch

Why wrong: Available pre-built container.

scikit-learn

Why wrong: Available pre-built container.

Use MultiWorkerMirroredStrategy with multiple workerPools

Why wrong: MultiWorkerMirroredStrategy is for multi-machine, not single-machine multi-GPU.

Use MirroredStrategy with two workerPoolSpecs, each with 2 GPUs

Why wrong: That would require MultiWorkerMirroredStrategy. MirroredStrategy works within a single process.

Use ParameterServerStrategy with a chief and a parameter server

Why wrong: ParameterServerStrategy is for asynchronous distributed training across machines.