CCNA Pmle Scaling Models Questions

Incremental learning with partial_fit is the correct approach because it lets a scikit-learn estimator update its parameters one chunk at a time. Reading the CSV in chunks with pandas.read_csv bounds memory to the chunk size rather than the full 2 TB. This keeps the single-model semantics, avoids distributed training, and fits within the 32 GB VM, directly satisfying all the stated constraints.

Decreasing the batch size directly reduces the memory required for activations and gradients, making it the simplest and most immediate way to resolve out-of-memory errors without altering the model architecture. It is a standard first step because it requires no code changes beyond a hyperparameter and often resolves OOM with minimal impact on convergence if adjusted with learning rate.

Mixed precision training and increasing batch size with learning rate scaling are both effective ways to improve training efficiency on V100 GPUs. Mixed precision leverages tensor cores for faster computation and reduced memory, while larger batches improve GPU utilization. These actions directly target performance and cost without altering the model architecture or data storage.

Vertex AI Feature Store (A) is correct because it provides a centralized repository for storing, serving, and sharing feature data with low-latency online serving and batch serving for training, ensuring consistency between training and serving through point-in-time lookups and feature value time-stamping. Cloud Dataflow (D) is correct because it is a fully managed stream and batch processing service based on Apache Beam, enabling real-time feature engineering from streaming data with exactly-once processing semantics and automatic scaling.

To conditionally run a step and fail the pipeline based on a metric, the engineer should use a condition to check the training step's status and a custom component that raises an exception if accuracy is too low. Vertex AI Pipelines conditions allow steps to run only if previous steps succeed, and raising an exception in a component causes the pipeline to fail. This combination provides the required control flow and failure semantics.

Using a pre-built Vertex AI container for scikit-learn with a custom training Python package ensures that the same preprocessing code runs during both training and serving, minimizing operational overhead. This approach leverages Vertex AI's managed infrastructure to handle scaling, monitoring, and consistency without requiring custom container maintenance.

Checkpoints must be written to a durable location, and in Vertex AI custom training the durable location is Cloud Storage. Passing a gs:// URI to the checkpoint directory makes TensorFlow persist each checkpoint through its GCS filesystem, and granting storage.objectAdmin to the training service account authorizes those writes. This allows both resuming training and retrieving the best model for deployment.

Pipeline parallelism requires explicit implementation in the training script, such as using `torch.distributed.pipeline.sync.Pipe` in PyTorch, to split the model layers across multiple GPUs. This is necessary when the model does not fit on a single GPU, and Vertex AI does not automatically handle model parallelism—it must be coded by the user.

The most likely cause of low TPU utilization is a data pipeline bottleneck, where the TPU spends a significant amount of time idle waiting for the next batch of data to be loaded and preprocessed. TPU v3 pods are designed for high-throughput matrix operations and can process data far faster than a typical CPU-based data loader can supply it, especially if the data pipeline uses inefficient I/O, lacks prefetching, or has insufficient workers. This mismatch starves the TPU, leading to low utilization and slower training.

Vertex AI Vizier is a hyperparameter tuning service that uses Bayesian optimization to efficiently search the hyperparameter space. With a limited budget of 50 trials, Bayesian optimization is more sample-efficient than random or grid search, making it the best choice.

A is correct because TPU pods require a startup script (e.g., `xla_lib.sh`) to initialize the XLA runtime, configure the TPU mesh, and set environment variables like `XRT_TPU_CONFIG`. Without this script, the TPU devices will not be discoverable by the PyTorch/XLA process, causing the training to fail with device-not-found errors.

For a custom container to work with Vertex AI, it must listen on the port specified by the AIP_HTTP_PORT environment variable and have access to the model artifacts. These are the two essential steps. Other aspects like health endpoints, SDK usage, or non-root user are optional or best practices but not mandatory for the container to serve predictions on Vertex AI.

The simplest and most efficient way to deploy a standard TensorFlow model on Vertex AI for online predictions is to export it as a SavedModel and use the pre-built TensorFlow Serving container. This requires no custom serving code, leverages Vertex AI's managed infrastructure, and provides low-latency predictions. It also integrates with Vertex AI Model Registry for versioning and monitoring.

Quantisation to INT8 reduces the precision of model weights and activations from 32-bit floating point to 8-bit integers, cutting memory usage by approximately 75% (4x compression). This directly addresses the 80% size reduction target while often preserving accuracy within 1-2% through careful calibration and scaling, making it a primary technique for edge deployment.

The a2-highgpu-4g machine type is purpose-built for GPU workloads and provides exactly 4 NVIDIA A100 GPUs on a single machine, matching the requirement precisely. Vertex AI supports this machine type for custom training with pre-built containers, so the data scientist can select it directly in the training configuration. Choosing it avoids over-provisioning or mismatching GPU counts.

In TensorFlow distributed training with MultiWorkerMirroredStrategy, the only required environment variable is `TF_CONFIG`. It provides the cluster topology and task identity, enabling gRPC communication between workers. The distribution strategy is defined in code, not via an environment variable. `TF_DISTRIBUTED_STRATEGY` is not a standard TensorFlow environment variable.

Vertex AI custom containers for prediction allow you to bundle the scikit-learn model, custom preprocessing code, and an HTTP server such as Flask. This keeps preprocessing and inference in the same process, reducing latency, and Vertex AI handles scaling and infrastructure management. Pre-built containers do not support arbitrary preprocessing, and batch prediction or model conversion would not meet the low-latency online requirement.

Vertex AI Training supports spot VMs (preemptible instances) for cost savings, and periodic checkpointing to Cloud Storage ensures that training can resume from the last saved state if a VM is preempted, allowing the job to complete within a bounded time despite interruptions.

Vertex AI JumpStart supports fine-tuning of foundation models like Gemma, allowing users to adapt pre-trained models to their specific datasets. This capability is built into JumpStart's managed environment, which handles the underlying infrastructure for training and deployment.

Cloud Dataflow is correct because it provides unified stream and batch processing with exactly-once semantics, enabling low-latency feature computation from streaming data. It integrates natively with Vertex AI Feature Store for online serving, ensuring features are updated within the required 5-second SLA.

Post-training quantization to INT8 reduces model weights and activations from 32-bit floats to 8-bit integers, cutting model size by roughly 4x and speeding up inference on edge hardware with minimal accuracy loss. It requires no retraining and works with most trained models. This makes it the best fit for limited-compute edge deployment.

Vertex AI custom training jobs use worker_pool_specs to define the machine type, container image, and accelerator configuration. The correct configuration uses the pre-built PyTorch GPU container and specifies machine_type='n1-standard-4', accelerator_type='NVIDIA_TESLA_V100', and accelerator_count=1 in the worker pool spec. This is the documented Vertex AI pattern for attaching a single V100 GPU.

In PyTorch DDP on Vertex AI, the number of workers equals the total number of processes across all nodes, which is nodes × GPUs per node. With 4 nodes and 4 GPUs each, the total is 16 workers, so the TF_CONFIG-equivalent worker count should be 16. Each GPU runs one DDP process, and all 16 processes participate in the all-reduce synchronization.

tf.Transform is a library built on Apache Beam that allows you to define preprocessing functions (like scaling and bucketizing) once and then apply them consistently in both training and serving. Dataflow provides a scalable, fully managed runner for Apache Beam pipelines, making it ideal for processing terabyte-scale datasets. This combination ensures that the exact same transformations are used during training and serving, avoiding training-serving skew.

When a model does not fit on a single GPU, model parallelism is required because it partitions the model itself across devices. Pipeline parallelism is a specific model parallelism technique that splits the model into stages across GPUs and pipelines micro-batches to maintain utilization, making it the appropriate approach for training a very large model on multiple GPUs in one machine.

Using a custom prediction routine with a preprocessor allows you to encapsulate preprocessing and model logic in a single package, ensuring consistency between training and serving. Vertex AI manages the infrastructure, providing low-latency and scalability. This is the most efficient and maintainable approach for scikit-learn models with custom preprocessing.

Wrapping preprocessing and the model into a single scikit-learn Pipeline ensures that the exact same transformations are applied during both training and serving. This eliminates training-serving skew and simplifies deployment. Vertex AI supports deploying custom models with custom prediction routines, but using a native scikit-learn Pipeline is more straightforward and less error-prone, as it encapsulates all steps and can be serialized as one artifact.

tf.distribute.MirroredStrategy is designed for synchronous, data-parallel training across multiple GPUs on a single machine. It replicates the model on each GPU, splits each batch across replicas, and uses all-reduce (via NCCL) to aggregate gradients, which is exactly the scenario described: 8 GPUs on one machine. This is the canonical strategy for single-node multi-GPU TensorFlow training.

The job fits in memory but fails on the default Vertex AI worker configuration, which points to insufficient RAM or vCPUs on the training machine. Choosing a larger machine type and an adequate boot disk gives the scikit-learn process the resources it needs. Distributed training is unnecessary and would require changing the algorithm, while preemptible VMs reduce reliability rather than improve it.

Bundling preprocessing and model inference into one custom container that follows the Vertex AI HTTP prediction contract keeps the request path short and self-contained, which minimizes latency. Deploying model versions to the same endpoint with a traffic split enables canary or rolling updates, so new versions can be validated with a small share of live traffic before full promotion, achieving zero-downtime releases.

Using TensorFlow Transform (tf.Transform) to define preprocessing ensures that the same transformations are applied during both training and serving. The transform function is included in the model graph, so the serving container automatically applies the same logic. This eliminates manual replication and reduces the risk of training-serving skew.

When a model is too large to fit on a single GPU, model parallelism is required to split the model's layers across multiple devices. Pipeline parallelism is a specific form of model parallelism that partitions layers into stages and pipelines micro-batches across devices, enabling training of models like a 7B-parameter LLM across multiple GPUs. This is the correct approach when memory, not throughput, is the binding constraint.

When the input pipeline is the bottleneck, the GPU waits for data. Using tf.data with prefetching and parallel extraction overlaps CPU preprocessing with GPU computation, ensuring the GPU is continuously fed. This is the standard TensorFlow optimization for such scenarios and directly improves GPU utilization.

TorchScript allows you to trace or script a PyTorch model, including preprocessing operations like tokenization and normalization, into a single serialized artifact. By embedding preprocessing in the TorchScript model, the entire pipeline runs in one forward pass on the Vertex AI endpoint, eliminating extra service calls and reducing latency. This is the standard approach for consolidating preprocessing with a PyTorch model for real-time inference.

In MirroredStrategy, all GPUs must finish their forward/backward pass before the all-reduce gradient synchronization can occur, so throughput is bounded by the slowest replica plus the sync overhead. With a small batch size, each GPU's compute time is very short, so the fixed cost of the NCCL all-reduce (and the sync wait) dominates the step time, making multi-GPU training slower than single-GPU. Increasing the per-replica batch size amortizes the synchronization cost over more compute, restoring scaling efficiency.

Post-training quantization to INT8 reduces model size by converting 32-bit floating-point weights and activations to 8-bit integers, which also speeds up inference on edge devices with integer-optimized hardware. This technique typically maintains accuracy within 1-2% of the original model while significantly lowering memory footprint and latency.

Versioning pipeline components and using Vertex AI Metadata are key practices for reproducibility and artifact tracking. Versioning ensures that the exact code and dependencies are captured, while Metadata automatically records parameters, metrics, and artifacts for each run, providing lineage and enabling reproducibility.

For batch prediction with custom preprocessing, a custom container is required because pre-built containers do not include your code. Vertex AI Batch Prediction natively supports BigQuery as both input and output, so you can avoid data movement. This method is efficient, scalable, and integrates with Vertex AI's managed infrastructure.

Freezing the earlier layers (which capture general features) and only training the later layers is a common transfer learning approach for small datasets, reducing overfitting.

TensorFlow Transform outputs a SavedModel that contains the preprocessing graph. This can be exported as a transform_fn and embedded in the serving model, ensuring consistency between training and serving.

The correct approach is to attach a custom service account with least-privilege Cloud Storage roles to the Vertex AI training job. Vertex AI training jobs automatically use the credentials of the attached service account via the metadata server, so no key files are needed. This ensures secure, short-lived authentication and follows Google Cloud best practices for access control.

Post-training quantization to INT8 is the correct first step because it directly reduces the model size by approximately 4x (from 32-bit floats to 8-bit integers) and speeds up inference on edge devices by leveraging integer-optimized hardware (e.g., ARM NEON or Qualcomm Hexagon). This technique requires no retraining and typically yields minimal accuracy loss for most TensorFlow models, making it the fastest path to deploy on resource-constrained devices.

Post-training quantization with TensorFlow Lite converts a trained model's weights from 32-bit floats to 8-bit integers, reducing model size by ~4x and speeding up inference on edge devices with minimal accuracy loss. TensorFlow Lite is purpose-built for edge deployment, making this the most direct and practical technique for the stated constraint.

Vertex AI JumpStart provides pre-built foundation models in Model Garden that can be deployed directly to a Vertex AI endpoint with minimal configuration. Selecting the model from Model Garden and deploying it to an endpoint is the standard JumpStart workflow for getting a foundation model into production for tasks like text summarization. No training, containerization, or batch export is required because JumpStart handles the deployment plumbing.

To optimize training cost and performance, the engineer should use accelerators and ensure the input pipeline is efficient, and store data in the same region as the training job. These actions reduce training time and avoid unnecessary network overhead. Other options like using broad service accounts or increasing steps do not contribute to efficiency and may introduce security or cost issues.

Communication issues in multi-node PyTorch DDP training often stem from incorrect MASTER_ADDR or MASTER_PORT settings, or network connectivity problems. These variables are critical for establishing the rendezvous point for all processes. Checking them first is a logical diagnostic step before considering other changes, as they are common causes of hangs and synchronization failures.

Dataflow with Apache Beam is the purpose-built managed service for streaming ETL on Google Cloud, natively integrating with Pub/Sub as a source and Vertex AI Feature Store as a sink. Beam's windowing, watermarks, and exactly-once semantics handle the large-scale, continuous feature computation required for online serving. It provides autoscaling and managed infrastructure, which is exactly what a production streaming feature pipeline needs.

When a model requires custom preprocessing that is not part of the model artifact itself, the most reliable way to serve it on Vertex AI is to build a custom container. The container can include the model, the preprocessing code, and all dependencies, ensuring consistent behavior between training and serving. Vertex AI then deploys this container as a Model resource and makes it available via an Endpoint.

When a single model's layers exceed one GPU's memory, the model itself must be partitioned across devices — this is model parallelism. Manual device placement with tf.device('/GPU:0'), tf.device('/GPU:1'), etc. is the TensorFlow-native way to assign specific layers or operations to specific GPUs, splitting the model across them.

Bayesian optimization (GP bandit) is Vizier's default and most sample-efficient algorithm, using a Gaussian Process surrogate model to balance exploration and exploitation across the 5-dimensional hyperparameter space. With only 50 trials over a large search space, it converges on promising regions far faster than uninformed methods. This makes it the best choice for maximizing the chance of finding the optimal configuration within a limited budget.

To serve low-latency predictions while allowing online embedding updates, the model should run in a custom container that can read and refresh the embedding table from an external store. This keeps the model deployed while the embeddings change independently. The other options either do not update internal embeddings, require redeployment, or are batch-oriented and therefore unsuitable for low-latency online serving.

Vertex AI Endpoints provide autoscaling to dynamically adjust replicas based on traffic. By enabling autoscaling with CPU utilization as the metric, the endpoint can scale out during bursts and scale in during low traffic, ensuring low latency and cost efficiency. This is the correct way to handle variable online prediction traffic.

MirroredStrategy is TensorFlow's default single-machine, multi-GPU strategy. It replicates the model on each GPU and synchronizes gradients via all-reduce, and it can be enabled with minimal code changes (typically just wrapping model creation in strategy.scope()), making it the right choice for scaling on one Vertex AI machine.

The out-of-memory error occurs because the training script attempts to load the entire 500 GB dataset into memory. Using TFRecord and tf.data allows streaming data in batches, so memory usage remains bounded. This is the standard method for scaling data input on Vertex AI and requires minimal changes to the training code.

Hugging Face's Trainer API exposes mixed precision through the TrainingArguments parameter fp16=True (or bf16=True for bfloat16). Setting this flag enables automatic mixed precision (AMP) via PyTorch's torch.cuda.amp, which casts eligible operations to FP16 while keeping master weights in FP32 for numerical stability. This is the standard, supported way to accelerate fine-tuning on GPU without rewriting the model.

Bayesian optimisation (option B) is the correct choice because it builds a probabilistic surrogate model of the objective function and uses an acquisition function to balance exploration and exploitation, making it highly sample-efficient. With only 50 trials, Bayesian optimisation maximises the probability of finding the global optimum by focusing trials on the most promising hyperparameter regions, unlike random or grid search which waste trials on unpromising areas.

When a transformer model is too large to fit on a single GPU, model parallelism (specifically pipeline parallelism) is required because it splits the model's layers across multiple devices, with each device holding a subset of the model's parameters. Data parallelism (DDP) replicates the entire model on each GPU, which fails if the model exceeds a single GPU's memory. Pipeline parallelism allows training very large models by partitioning the model into stages and passing activations and gradients sequentially between devices.

Dataflow (Apache Beam) is ideal for processing large-scale batch and streaming data. It can read from BigQuery, perform windowing computations, and write to Feature Store's online store. Cloud Functions have timeouts, Cloud Composer is not optimal for streaming, and BigQuery scheduled queries are not designed for streaming-feature computation.

For custom preprocessing with non-TensorFlow models like XGBoost, a custom container is the most reliable way to ensure that the exact same preprocessing is applied at serving time. You can embed the training statistics and logic in the container, maintaining consistency and simplifying client requests.

The most efficient approach is to materialize the BigQuery data in Cloud Storage once and then have the training container read those files for every epoch. This removes repeated BigQuery scans, reduces cost and latency, and lets the training job use high-throughput file readers. The other choices either keep the expensive repeated reads or add complexity without fixing the core bottleneck.

Bayesian optimization (the default algorithm in Vertex AI Vizier) is the correct choice because it explicitly balances exploration and exploitation by building a probabilistic model of the objective function and using an acquisition function to select the next hyperparameter configuration. With a budget of 50 trials, this algorithm efficiently converges to optimal regions while still exploring uncertain areas, making it ideal for tuning custom training jobs where each trial is computationally expensive.

Fine-tuning a pre-trained model on your custom dataset is the most appropriate approach when you have a small dataset and need to adapt the model to new categories. This leverages transfer learning, where the pre-trained weights are used as a starting point and updated with your data.

Vertex AI supports spot VMs (the successor to preemptible VMs) by setting the 'spot' field to true in the machine spec of the worker pool. Spot VMs offer up to 60-91% discounts and are suitable for fault-tolerant jobs with checkpointing, since they can be preempted with 30 seconds' notice. This is the documented, supported configuration for cost-optimized training.

Post-training INT8 quantization reduces model size by approximately 75% (from ~440 MB to ~110 MB for BERT-Base) and accelerates inference on edge devices via integer arithmetic, easily meeting the 500 MB and 50 ms constraints. TensorFlow Lite provides hardware-optimized kernels for ARM CPUs and NPUs, making it ideal for edge deployment without requiring retraining.

tf.Transform requires Apache Beam for execution, which on GCP is typically run on Dataflow. The processed data and transform artifacts are stored in Cloud Storage.

The BigQuery Storage API is designed for high-performance data reading, offering parallel streams and efficient data transfer. It avoids the overhead of exporting data to Cloud Storage or loading into memory all at once. This makes it the most efficient and cost-effective approach for training on large BigQuery datasets.

Bayesian optimization is the default and best-suited algorithm in Vertex AI Vizier for mixed parameter spaces containing both categorical and continuous parameters. It builds a probabilistic surrogate model of the objective function and uses an acquisition function to intelligently select the next hyperparameter configuration, handling categorical and continuous dimensions natively. This makes it far more sample-efficient than random or grid search when the search space is mixed and potentially high-dimensional.

Low GPU utilization with many small files typically points to an I/O-bound data pipeline. Cloud Storage has high per-file access latency, so reading hundreds of thousands of small JPEGs individually creates a bottleneck. Converting to a sharded format like TFRecord or WebDataset enables efficient sequential reads, and combining with prefetching and larger batches ensures the GPU is continuously fed, improving utilization and training speed.

To ensure features are available for both training and serving with low latency, the featurestore must have online serving enabled. Vertex AI Feature Store provides both online serving (low-latency reads for real-time predictions) and offline serving (batch reads for training). Enabling online serving allows the same feature values to be served at low latency during prediction, ensuring consistency between training and serving.