CCNA Scaling Prototypes into ML Models Questions

Decreasing the batch size is the simplest and most direct approach to reduce GPU memory usage for a custom training job. It linearly reduces the memory needed for activations and gradients. While mixed precision can also reduce memory, it may introduce numerical precision issues and only works on compatible hardware, so decreasing batch size should be attempted first.

In Vertex AI Vizier, parallel trial execution is configured by setting the 'parallelTrialCount' field in the study configuration. The maxParallelTrials is not a direct field; instead, the StudySpec contains a StudyJobConfig with parallelTrialCount. Setting parallelTrialCount to 10 allows up to 10 trials to run concurrently.

Vertex AI Feature Store (A) is correct because it provides a centralized repository for storing, serving, and sharing feature data with low-latency online serving and batch serving for training, ensuring consistency between training and serving through point-in-time lookups and feature value time-stamping. Cloud Dataflow (D) is correct because it is a fully managed stream and batch processing service based on Apache Beam, enabling real-time feature engineering from streaming data with exactly-once processing semantics and automatic scaling.

The correct artifact to save after analyzing training data with tf.Transform is the `transform_fn` SavedModel. This SavedModel encapsulates the exact preprocessing logic (e.g., scaling, normalization, vocabulary mapping) computed from the training dataset, ensuring that the same transformations are applied consistently during both training and serving. Without this artifact, the serving pipeline would need to recompute or duplicate the transformation logic, risking skew between training and inference.

Using a pre-built Vertex AI container for scikit-learn with a custom training Python package ensures that the same preprocessing code runs during both training and serving, minimizing operational overhead. This approach leverages Vertex AI's managed infrastructure to handle scaling, monitoring, and consistency without requiring custom container maintenance.

Spot VMs are cheaper but can be preempted; using checkpointing and saving to a durable storage (like GCS) allows recovery. Also, using a single node with multiple GPUs may be cheaper than multiple nodes.

Quantisation-aware training (QAT) simulates quantisation during training, allowing the model to adapt, resulting in higher accuracy than post-training quantisation. Post-training quantisation is simpler but may lose accuracy.

Pipeline parallelism requires explicit implementation in the training script, such as using `torch.distributed.pipeline.sync.Pipe` in PyTorch, to split the model layers across multiple GPUs. This is necessary when the model does not fit on a single GPU, and Vertex AI does not automatically handle model parallelism—it must be coded by the user.

The most likely cause of low TPU utilization is a data pipeline bottleneck, where the TPU spends a significant amount of time idle waiting for the next batch of data to be loaded and preprocessed. TPU v3 pods are designed for high-throughput matrix operations and can process data far faster than a typical CPU-based data loader can supply it, especially if the data pipeline uses inefficient I/O, lacks prefetching, or has insufficient workers. This mismatch starves the TPU, leading to low utilization and slower training.

For GPU-accelerated prediction, the T4 GPU is cost-effective and provides good performance for inference. Autoscaling with a minimum of 1 instance ensures availability while allowing the service to scale down when not in use. A100 is more expensive; batch prediction is for asynchronous large-scale jobs; manual scaling may lead to over-provisioning.

Bayesian optimization is the most efficient algorithm for hyperparameter tuning when the number of trials is limited. It builds a probabilistic model of the objective function and selects promising configurations.

A is correct because TPU pods require a startup script (e.g., `xla_lib.sh`) to initialize the XLA runtime, configure the TPU mesh, and set environment variables like `XRT_TPU_CONFIG`. Without this script, the TPU devices will not be discoverable by the PyTorch/XLA process, causing the training to fail with device-not-found errors.

PipelineMirroredStrategy combines model parallelism (splitting the transformer layers across multiple GPUs) with pipeline parallelism to handle models that exceed single GPU memory. This strategy partitions the model into stages, each placed on a different GPU, and uses micro-batching to keep all GPUs busy, which is essential for large transformer models that cannot fit into a single GPU's memory.

Vertex AI JumpStart provides one-click deployment of foundation models like Llama, Gemini, etc. Model Garden is for discovery, but JumpStart directly deploys.

For fine-tuning with a sufficiently large dataset, it is common to unfreeze all layers to adapt the model to the new task. Freezing many layers is typical for very small datasets. With 100k examples, full fine-tuning is appropriate.

Quantisation to INT8 reduces the precision of model weights and activations from 32-bit floating point to 8-bit integers, cutting memory usage by approximately 75% (4x compression). This directly addresses the 80% size reduction target while often preserving accuracy within 1-2% through careful calibration and scaling, making it a primary technique for edge deployment.

In TensorFlow distributed training with MultiWorkerMirroredStrategy, the only required environment variable is `TF_CONFIG`. It provides the cluster topology and task identity, enabling gRPC communication between workers. The distribution strategy is defined in code, not via an environment variable. `TF_DISTRIBUTED_STRATEGY` is not a standard TensorFlow environment variable.

Vertex AI Training supports spot VMs (preemptible instances) for cost savings, and periodic checkpointing to Cloud Storage ensures that training can resume from the last saved state if a VM is preempted, allowing the job to complete within a bounded time despite interruptions.

Vertex AI JumpStart supports fine-tuning of foundation models like Gemma, allowing users to adapt pre-trained models to their specific datasets. This capability is built into JumpStart's managed environment, which handles the underlying infrastructure for training and deployment.

Transfer learning typically involves loading a pre-trained model (e.g., ResNet50 from Keras Applications) without the top classification layer, freezing all layers, adding new trainable layers on top, and then training. The base model's layers are frozen (trainable=False). After initial training, one can optionally unfreeze some top layers.

Cloud Dataflow is correct because it provides unified stream and batch processing with exactly-once semantics, enabling low-latency feature computation from streaming data. It integrates natively with Vertex AI Feature Store for online serving, ensuring features are updated within the required 5-second SLA.

Post-training quantization (e.g., INT8) is the easiest and most effective method for reducing model size and latency on edge devices. Quantization-aware training can yield better accuracy but is more complex. Pruning and distillation also help, but quantization often gives the best trade-off.

In Vertex AI, worker_pool_specs define machine types and accelerators. Using a pre-built container for PyTorch 1.12 with image_uri 'us-docker.pkg.dev/vertex-ai/training/pytorch-gpu.1-12:latest' and specifying worker_count=1, machine_type='n1-standard-4' (or similar), and accelerator_type='NVIDIA_TESLA_V100' with count=1 sets up the job correctly.

In PyTorch DDP on Vertex AI, each GPU typically runs one process. With 4 nodes each with 4 GPUs, the world size is 16 (total GPUs). Vertex AI distributed training uses TF_CONFIG to set up the cluster; for PyTorch, you need to set the number of workers to the total number of processes, which is 16.

TensorFlow Transform (tf.Transform) allows you to define preprocessing pipelines that are applied consistently during training and serving. It computes statistics over the full dataset and creates a TensorFlow graph for serving.

When a model is too large for a single GPU, model parallelism (specifically pipeline parallelism) splits the model across GPUs. Data parallelism would require each GPU to hold the full model, which is not possible.

Bayesian optimization is the default and recommended algorithm for most use cases as it balances exploration and exploitation efficiently. For a budget of 50 trials, Bayesian optimization is suitable.

Options A and C are correct because TPU pod slices on Vertex AI require setting TF_CONFIG for distributed training (A) and specifying the accelerator type as TPU_V3 with the appropriate topology (C). Option D is not required: Vertex AI provides pre-built containers for TensorFlow that already include TPU support, so a custom container is unnecessary. Using a custom container would be an extra step not needed for this scenario.

When fine-tuning a pre-trained model on a small dataset that is very similar to the original training data, the safest strategy to avoid overfitting is to freeze all layers and train only the classifier head. This preserves the rich, general-purpose feature representations learned from the original large dataset, while allowing the final classification layer to adapt to the new task. Training the entire model or unfreezing many layers on a small dataset would risk overfitting because the model would have too many parameters to update relative to the limited new samples.

MirroredStrategy synchronously updates gradients across GPUs. The overhead of gradient synchronization (all-reduce) can become a bottleneck if the model is small or the network between GPUs is slow. This is a common issue.

Post-training quantization to INT8 reduces model size by converting 32-bit floating-point weights and activations to 8-bit integers, which also speeds up inference on edge devices with integer-optimized hardware. This technique typically maintains accuracy within 1-2% of the original model while significantly lowering memory footprint and latency.

Vertex AI JumpStart provides one-click deployment of foundation models and ML solutions. It allows deploying pre-trained models without coding.

Vertex AI Prediction has a request size limit (1.5 MB). Using a custom container with a ModelServer (e.g., TensorFlow Serving) behind an HTTP load balancer bypasses this limit and allows large payloads.

In Vertex AI Vizier, continuous parameters use DOUBLE type (can be scaled log), integer parameters use INTEGER, and categorical parameters use CATEGORICAL. The scale type for learning rate should be UNIT_LOG_SCALE for log scale.

To customize the output layer, you need a custom container that loads both the fine-tuned base model and your custom head. The model must be registered in Vertex AI Model Registry for deployment.

Freezing the earlier layers (which capture general features) and only training the later layers is a common transfer learning approach for small datasets, reducing overfitting.

TensorFlow Transform outputs a SavedModel that contains the preprocessing graph. This can be exported as a transform_fn and embedded in the serving model, ensuring consistency between training and serving.

Post-training quantization to INT8 is the correct first step because it directly reduces the model size by approximately 4x (from 32-bit floats to 8-bit integers) and speeds up inference on edge devices by leveraging integer-optimized hardware (e.g., ARM NEON or Qualcomm Hexagon). This technique requires no retraining and typically yields minimal accuracy loss for most TensorFlow models, making it the fastest path to deploy on resource-constrained devices.

Post-training quantization (e.g., INT8) reduces model size and speeds up inference with minimal accuracy loss. It is the simplest method for deployment on edge devices.

JumpStart provides one-click deployment of foundation models from Model Garden. You select a model and deploy it to an endpoint. No custom training or container building is needed.

Model parallelism splits the model itself across devices, as opposed to data parallelism which replicates the model. For TensorFlow, this is typically achieved using device placement or tf.distribute.experimental.ParameterServerStrategy with manual partitioning, but for splitting layers, tf.distribute.experimental.DeviceAssignment or manual tf.device is used.

Increasing the number of parallel trials can help explore more of the hyperparameter space, but the specified number of max trials must be high enough. Reducing parallel trials may slow exploration. Bayesian optimisation benefits from more trials.

MirroredStrategy distributes training across GPUs on a single machine with minimal code changes — just wrap model creation and compile inside the strategy scope. MultiWorkerMirroredStrategy is for multi-machine, not single-machine multi-GPU.

Vertex AI JumpStart (now part of Model Garden) provides one-click deployment and fine-tuning of foundation models. Model Garden is the broader model discovery hub, but JumpStart is the feature for quick start.

Vertex AI JumpStart is the correct choice because it provides a managed environment with pre-built, optimized containers for popular models like Vision Transformers, enabling one-click deployment and fine-tuning with minimal setup. It abstracts away infrastructure management, allowing the data scientist to quickly experiment without configuring custom training scripts or environments.

Hugging Face Trainer supports mixed precision via the fp16 argument. Set it to True in the TrainingArguments. No need to modify the model architecture or use a custom container.

JumpStart fine-tuning typically involves selecting a model, connecting to your dataset (often via BigQuery), configuring training, and launching. The process is largely GUI-based.

Bayesian optimisation (option B) is the correct choice because it builds a probabilistic surrogate model of the objective function and uses an acquisition function to balance exploration and exploitation, making it highly sample-efficient. With only 50 trials, Bayesian optimisation maximises the probability of finding the global optimum by focusing trials on the most promising hyperparameter regions, unlike random or grid search which waste trials on unpromising areas.

Weight pruning (B) is suitable because it removes redundant connections (weights) from the neural network, reducing the model size and computational cost while often preserving accuracy if done gradually. Post-training float16 quantization (D) converts model weights from float32 to float16, halving the storage size with minimal accuracy loss, and is directly supported by TensorFlow Lite for edge deployment.

When a transformer model is too large to fit on a single GPU, model parallelism (specifically pipeline parallelism) is required because it splits the model's layers across multiple devices, with each device holding a subset of the model's parameters. Data parallelism (DDP) replicates the entire model on each GPU, which fails if the model exceeds a single GPU's memory. Pipeline parallelism allows training very large models by partitioning the model into stages and passing activations and gradients sequentially between devices.

Dataflow (Apache Beam) is ideal for processing large-scale batch and streaming data. It can read from BigQuery, perform windowing computations, and write to Feature Store's online store. Cloud Functions have timeouts, Cloud Composer is not optimal for streaming, and BigQuery scheduled queries are not designed for streaming-feature computation.

Bayesian optimization (the default algorithm in Vertex AI Vizier) is the correct choice because it explicitly balances exploration and exploitation by building a probabilistic model of the objective function and using an acquisition function to select the next hyperparameter configuration. With a budget of 50 trials, this algorithm efficiently converges to optimal regions while still exploring uncertain areas, making it ideal for tuning custom training jobs where each trial is computationally expensive.

Vertex AI supports spot VMs (formerly preemptible) for training. By setting 'spot' to true, you use preemptible instances at a discount. The training must handle preemption via checkpointing.

Enabling early stopping in Vertex AI hyperparameter tuning terminates poorly performing trials early, saving the trial budget for more promising hyperparameter configurations. This is critical when the objective is to minimize validation loss with a limited budget of 100 trials, as it prevents wasting resources on suboptimal runs and allows the search to focus on the most promising regions of the hyperparameter space.

Post-training INT8 quantization reduces model size by approximately 75% (from ~440 MB to ~110 MB for BERT-Base) and accelerates inference on edge devices via integer arithmetic, easily meeting the 500 MB and 50 ms constraints. TensorFlow Lite provides hardware-optimized kernels for ARM CPUs and NPUs, making it ideal for edge deployment without requiring retraining.

tf.Transform requires Apache Beam for execution, which on GCP is typically run on Dataflow. The processed data and transform artifacts are stored in Cloud Storage.

Tf.Transform is specifically designed to handle full-pass preprocessing (e.g., computing min/max, vocabularies) that requires seeing the entire dataset. By defining a `preprocessing_fn` and applying it via `tft.beam.analyze_and_transform`, the transform graph is exported as a SavedModel, which can be loaded during serving to ensure identical preprocessing logic. This guarantees consistency between training and inference, which is critical for production ML pipelines.

Bayesian optimization can handle mixed parameter types. Grid search cannot handle continuous parameters efficiently; random search can, but Bayesian is more sample-efficient.

Vertex AI Feature Store supports online serving (low latency) and offline serving (batch). For low latency, they must enable online serving. The online store uses Bigtable for low-latency lookups.

Pipeline parallelism splits layers across devices; model parallelism (operator-level) splits individual operations. Data parallelism replicates the model. ParameterServer is a form of distributed training, not model splitting.

Vertex AI automatically streams all container stdout and stderr to Cloud Logging (Logs Explorer). When a custom container exits with a non-zero status, the detailed error messages, stack traces, and application logs are captured there, making it the primary and most comprehensive debugging tool. Cloud Logging provides structured, searchable logs without requiring direct access to the container.

Vertex AI hyperparameter tuning requires specifying the search algorithm (Bayesian, grid, or random) to determine how the hyperparameter space is explored. Bayesian optimization is the default and most efficient for continuous spaces, while grid search is exhaustive and random search is simple. Without an algorithm, Vertex AI cannot decide how to sample trials.

Vertex AI provides pre-built containers for scikit-learn with the prefix 'us-docker.pkg.dev/vertex-ai/training/scikit-learn-cpu.0.23-0'. The others are for different frameworks.

TPU training requires specifying the TPU type in the worker pool spec and using tf.distribute.TPUStrategy in code. The pre-built container supports TPUs. Runtime version must include TPU support.