NCA-GENL Core Machine Learning and AI Knowledge Practice Test 4 — 15 Questions

Question 1

When training a model with a very large dataset, which approach provides the best balance between computational efficiency and model convergence?

Accepted Answer

Mini-batch gradient descent.. Mini-batch gradient descent strikes an ideal balance by processing small subsets of the data (batches) at a time. This provides the efficiency of vectorization on GPU hardware while maintaining enough stochasticity to help the optimizer avoid poor local minima. Compared to full batch (too slow) or single-sample stochastic descent (too noisy), mini-batching is the industry standard for scaling training to large datasets while achieving optimal convergence.

Answer

Full-batch gradient descent.

Answer

Stochastic gradient descent with batch size 1.

Answer

Gradient descent without any data shuffling.

Question 2

Refer to the exhibit. Which technique is most effective for preventing the reported NaN error during model training?

Accepted Answer

Gradient Clipping. The exhibit shows a classic case of exploding gradients, where the gradient norm spikes and leads to non-finite weight values. Gradient clipping is the standard industry practice to mitigate this, as it caps the gradient magnitude during backpropagation. Implementing this ensures stability in deep networks, preventing training collapses that waste expensive GPU compute resources and time in large-scale AI development cycles.

Answer

Using a larger batch size

Answer

Enabling Layer Normalization

Answer

Changing the optimizer to SGD

Question 3

Which of the following describes the purpose of a validation set in machine learning?

Accepted Answer

To tune hyperparameters and prevent overfitting. The validation set serves as an independent dataset used to tune hyperparameters and monitor model performance during training. It provides an unbiased evaluation of the model's generalization capabilities, allowing engineers to prevent overfitting and select the best model version before final testing. Distinguishing between training, validation, and test data is fundamental to ensuring models perform reliably in production environments, avoiding common pitfalls in AI development.

Answer

To increase the total amount of training data available

Answer

To calculate the final accuracy on unseen production data

Answer

To perform backpropagation during the model training loop

Question 4

Which activation function is most commonly used in the hidden layers of deep neural networks to mitigate the vanishing gradient problem?

Accepted Answer

ReLU. The ReLU (Rectified Linear Unit) activation function is the industry standard for hidden layers because it avoids the saturation characteristic of sigmoid or tanh functions. By outputting zero for negative inputs and linear values for positive inputs, it maintains a gradient of one during backpropagation for active neurons. This allows gradients to flow through deep networks without shrinking exponentially, which is essential for training modern, deep architectures effectively.

Answer

Sigmoid

Answer

Softmax

Answer

Hyperbolic Tangent (tanh)

Question 5

Refer to the exhibit. The training loss is oscillating and failing to converge. What is the most likely immediate adjustment needed?

Accepted Answer

Reduce the learning rate. The oscillating loss and lack of convergence suggest the learning rate is too high. A learning rate of 0.01 is relatively aggressive for many deep learning tasks, causing the model to jump over the optimal minima. Reducing the learning rate is the standard first step to stabilize the training process, allowing the optimizer to settle into the local minimum more consistently and achieve better overall convergence results.

Answer

Increase the batch size to 1024

Answer

Switch to a larger model architecture

Answer

Switch to FP16 mixed-precision

Question 6

Which component of an NVIDIA Transformer Engine is specifically designed to accelerate training on supported GPUs by dynamically adjusting precision?

Accepted Answer

Transformer Engine. The NVIDIA Transformer Engine uses FP8 precision and dynamic scaling to drastically accelerate training for transformer-based models. By utilizing hardware-level support for mixed-precision, it maximizes throughput while maintaining accuracy. Understanding how these hardware-software co-optimizations function is critical for modern LLM development, as they allow for training larger models in less time and with lower power usage compared to standard FP32 or mixed-precision training workflows.

Answer

Tensor Cores

Answer

CUDA Streams

Answer

NCCL (NVIDIA Collective Communications Library)

Question 7

What is the primary function of the 'Attention' mechanism in Transformer models?

Accepted Answer

To enable the model to weigh the importance of different input tokens. The attention mechanism allows the model to compute weights that signify the relevance of different parts of the input sequence to one another, regardless of their distance. This captures long-range dependencies effectively, which traditional RNNs struggle with due to vanishing gradients over long sequences. Mastering this concept is key to understanding why transformers have become the dominant architecture for nearly all state-of-the-art generative AI and natural language processing applications.

Answer

To reduce the number of parameters in the model

Answer

To convert text into numerical vectors

Answer

To perform sequential processing of inputs

Question 8

Refer to the exhibit. Which strategy is most effective for resolving this memory error without changing the hardware?

Accepted Answer

Gradient Accumulation. The exhibit indicates that the GPU is nearly at capacity. Gradient accumulation is a standard technique that simulates a larger batch size by accumulating gradients over multiple smaller steps before performing a weight update. This allows the user to maintain the intended effective batch size while fitting the training within the physical memory limits of the GPU, ensuring the training process can proceed.

Answer

Increase the hidden layer size

Answer

Enable more data augmentation

Answer

Increase the number of training epochs

Question 9

Which THREE of the following factors are critical when choosing a foundation model for an enterprise generative AI application?

Accepted Answer

Model licensing and data privacy policies. Selecting a foundation model involves balancing technical performance, operational costs, and security requirements. Enterprise readiness requires understanding data privacy policies, licensing, and the ability to fine-tune the model for domain-specific tasks. Failing to evaluate these factors can lead to significant downstream issues, including legal risks, poor performance, or unmanageable infrastructure costs, making this selection process one of the most important steps in an AI project lifecycle.

Answer

The number of hidden layers in the model

Answer

The specific programming language used to build the model

Question 10

What is the primary role of 'Loss Scaling' when training deep learning models in FP16 precision?

Accepted Answer

To prevent gradient underflow in FP16 training. Loss scaling is essential because FP16 has a narrower dynamic range than FP32. Small gradient values can underflow to zero, causing the model to stop learning. By scaling the loss up before backpropagation, the gradients are kept within the representable range of FP16, and then scaled back down during the weight update, ensuring stable and effective training while maintaining the speed advantages of half-precision compute.

Answer

To increase the training speed of the GPU

Answer

To reduce the required GPU memory footprint

Answer

To normalize the input data distribution

Question 11

An enterprise machine learning team is training a large-scale transformer model on a cluster of NVIDIA A100 GPUs using mixed precision (FP16). During the initial training phase, the team notices sudden numerical underflow resulting in vanishing gradients and stalled loss convergence. Which optimization technique must be applied to mitigate this issue without sacrificing the memory-efficiency benefits of FP16?

Accepted Answer

Applying dynamic loss scaling to the loss value prior to backpropagation to shift gradients into the representational range.. Loss scaling multiplies the forward pass loss by a scaling factor to shift small gradient magnitudes into the representational range of the FP16 format, preventing underflow. This is essential in NVIDIA mixed-precision training because FP16 has a narrow dynamic range compared to FP32. Proper scaling prevents gradient values from truncating to zero while retaining half-precision throughput and memory footprint advantages on Tensor Cores.

Answer

Implementing gradient clipping by global norm to bound large gradient updates before parameter application.

Answer

Switching the entire training cluster to FP32 precision to guarantee maximum numerical stability and range.

Answer

Increasing the mini-batch size exponentially to smooth out noisy gradient estimates across distributed nodes.

Question 12

An AI researcher is fine-tuning a large language model and wants to minimize GPU memory consumption during training without altering the model's primary weight representations or introducing quantization error during inference. Which technique provides this capability by decomposing weight matrices into low-rank trainable adaptation matrices?

Accepted Answer

Low-Rank Adaptation (LoRA) freezing pre-trained weights and training injected rank decomposition matrices.. Low-Rank Adaptation (LoRA) freezes the original pre-trained model weights and injects trainable rank decomposition matrices into the architecture. This drastically reduces the number of trainable parameters and optimizer states in GPU memory during backpropagation, while allowing weights to be merged back into the original matrices for zero-latency deployment.

Answer

Post-Training Quantization (PTQ) converting weights from FP16 to INT8 integer representations.

Answer

Full-Parameter Fine-Tuning updating all weights across every transformer layer simultaneously using FP32 precision.

Answer

Knowledge Distillation transferring learned representations from a large teacher model to a smaller student network architecture.

Question 13

Refer to the exhibit. A machine learning engineer reviews the monitoring output from a two-GPU distributed training job running on an NVIDIA DGX system. GPU 0 shows low utilization despite high memory consumption, while GPU 1 shows high utilization and high memory consumption. What is the most likely root cause of this performance imbalance?

Accepted Answer

Imbalanced data distribution or pipeline synchronization stalls causing GPU 0 to wait idly while holding allocated tensors.. Data parallelism with uneven batch distribution or pipeline parallel bubble inefficiency can cause worker synchronization stalls. When one GPU finishes its workload or waits for synchronization barriers while processing uneven tensor sizes or asynchronous communication primitives, its utilization drops while memory remains allocated to tensors.

Answer

A PCIe bus bottleneck causing continuous memory swapping between host system RAM and GPU 0 device memory.

Answer

A hardware fault in the NVIDIA NVLink bridge connecting GPU 0 to the shared system NVSwitch fabric.

Answer

An incorrect CUDA driver version mismatch preventing GPU 0 from initializing its primary Tensor Core execution units.

Question 14

An ML engineer is deploying a Transformer-based inference service on an NVIDIA TensorRT-LLM runtime. To maximize inference throughput and reduce latency under heavy concurrent user traffic, the engineer needs to select the optimal decoding batching strategy. Which technique allows multiple incoming dynamic sequence requests to be batched together at the token level rather than waiting for entire sequences to complete?

Accepted Answer

Inflight batching (continuous batching) that schedules individual generation iterations at the token level across requests.. Inflight batching (also known as continuous batching) dynamically groups individual generation steps from different requests at each token generation iteration. This eliminates the idle padding overhead and waiting times associated with traditional static batching, maximizing GPU compute utilization in LLM serving.

Answer

Static request batching that groups fixed numbers of incoming inference prompts and pads them to equal sequence lengths.

Answer

Data-parallel replica sharding that duplicates the model across separate GPU devices to process independent streams.

Answer

Model parallelism that splits individual transformer weight matrices across multiple interconnected GPU devices.

Question 15

A data scientist is preparing a transformer-based language model for a text summarization task. She notices that the input sequences in her dataset vary widely in length, from a few tokens to several thousand. She decides to set a fixed maximum sequence length and pad shorter sequences with a special token. Which component of the transformer architecture is primarily responsible for handling the positional information of tokens in these sequences?

Accepted Answer

Positional encodings. Positional encodings are essential in transformer models because the self-attention mechanism is permutation-invariant. By adding positional encodings to token embeddings, the model gains awareness of token order, enabling it to handle sequences of varying lengths and padding effectively. This is critical for tasks like summarization where the meaning depends on word order.

Answer

Self-attention mechanism

Answer

Layer normalization

Answer

Feed-forward network