NCA-GENL Core Machine Learning and AI Knowledge Practice Test 2 — 15 Questions

Question 1

A research team is pre-training a large language model on a massive corpus of text. They want to ensure that the model learns useful representations and generalizes well to downstream tasks. Which TWO of the following techniques are commonly used during pre-training to improve the model's ability to capture contextual relationships and avoid overfitting? (Choose two.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Masked language modeling (MLM). Masked language modeling and causal language modeling are two fundamental pre-training objectives for large language models. MLM enables bidirectional context learning, while CLM enables autoregressive generation. Both are essential for learning rich representations that generalize to downstream tasks, and they inherently prevent overfitting by exposing the model to diverse contexts.

Answer

Batch normalization

Answer

Weight initialization with small random values

Answer

Dropout on input embeddings

Question 2

A researcher is training a transformer-based language model and wants to prevent the model from attending to future tokens during training. Which mechanism should be implemented in the self-attention layer?

Accepted Answer

Add a causal mask to the attention scores before the softmax operation.. To prevent attending to future tokens, a causal mask must be applied to the attention scores before softmax. This sets future positions to negative infinity, resulting in zero attention probability. Other techniques like layer normalization, dropout, or more heads do not enforce causality and are unrelated to masking future tokens.

Answer

Use a higher dropout rate on the attention weights.

Answer

Apply layer normalization before the self-attention computation.

Answer

Increase the number of attention heads.

Question 3

A team is deploying a large language model for real-time inference on NVIDIA GPUs. They want to reduce latency and increase throughput. Which TWO techniques are most effective for achieving these goals? (Choose two.)

Accepted Answer

Using NVIDIA TensorRT to optimize the model graph.. Quantization and TensorRT optimization are both highly effective at reducing latency and increasing throughput for LLM inference on NVIDIA GPUs. Quantization lowers precision to speed up computation and reduce memory bandwidth, while TensorRT optimizes the model graph for the specific GPU. Together, they provide substantial performance gains without altering the model's core architecture, making them the most suitable choices for real-time deployment.

Answer

Increasing the batch size beyond the GPU's memory capacity.

Answer

Enabling dynamic batching to group incoming requests.

Answer

Implementing a larger number of transformer layers.

Question 4

A team is deploying a large language model for real-time chatbot inference on NVIDIA GPUs. They need to reduce latency without sacrificing output quality. Which two techniques should they implement? (Choose two.)

Accepted Answer

Apply kernel fusion to combine multiple operations into a single GPU kernel.. Quantization to INT8 reduces memory bandwidth and leverages faster integer tensor core operations, while kernel fusion minimizes overhead and memory traffic. Both techniques are widely used in NVIDIA inference stacks to lower latency without retraining or degrading output quality. They directly address the real-time constraint by making each forward pass faster and more efficient.

Answer

Increase the batch size for each inference request.

Answer

Switch from FP16 to FP32 for all computations.

Answer

Increase the number of attention heads in each transformer layer.

Question 5

A researcher is training a large language model and notices the training loss plateaus early while validation loss increases. What is the most likely cause, and which action should be taken?

Accepted Answer

The model is overfitting; apply dropout or weight decay.. The symptoms described clearly indicate overfitting, where the model captures noise in the training set rather than generalizing to unseen data. In the context of large language models, this is a critical challenge. Implementing regularization techniques such as weight decay or dropout helps constrain model complexity, forcing it to learn more robust features rather than memorizing specific patterns, thereby improving overall model performance and generalizability.

Answer

The model is underfitting; increase the number of hidden layers.

Answer

The learning rate is too low; increase it to accelerate convergence.

Answer

The dataset is too small; reduce the batch size.

Question 6

In the context of transformer models, what is the purpose of the 'Attention Mask' during the training process?

Accepted Answer

It prevents the model from attending to future tokens in autoregressive models.. The attention mask is essential for transformer architectures to handle sequences of varying lengths and to enforce causal constraints. In tasks like sequence generation, it prevents the model from 'peeking' at future tokens, ensuring that the prediction at each position depends only on preceding tokens. This mechanism is critical for maintaining the autoregressive nature of models like GPT and ensuring correct model training.

Answer

It serves to reduce the number of parameters in the self-attention layer.

Answer

It optimizes the data movement between the GPU's L1 and L2 cache.

Answer

It performs weight pruning to compress the model size after training.

Question 7

What is the primary function of Layer Normalization in a transformer architecture?

Accepted Answer

To stabilize training by normalizing the inputs to each layer.. Layer normalization stabilizes the hidden state distributions across layers by normalizing inputs to have zero mean and unit variance. This prevents internal covariate shift and keeps activations within a stable range, allowing for faster convergence and deeper networks. In transformers, it is typically applied to the output of sub-layers, ensuring that subsequent computations remain numerically stable and well-conditioned for backpropagation.

Answer

To increase the total parameter count of the model.

Answer

To replace the need for weight initialization techniques.

Answer

To compress the model to fit on smaller GPU devices.

Question 8

Why is 'Warmup' used for the learning rate schedule during the initial phase of training large language models?

Accepted Answer

To prevent the optimizer from diverging due to large, noisy initial gradients.. Learning rate warmup is essential for training stability. At the beginning of training, weights are random, and gradients can be volatile. A low initial learning rate prevents the optimizer from making massive, potentially catastrophic updates that could diverge the model. Gradually increasing the rate allows the model to stabilize and find a more favorable region in the loss landscape, ensuring a smooth and reliable start to the training process.

Answer

To reduce the amount of data needed to reach convergence.

Answer

To automatically detect the optimal batch size for the hardware.

Answer

To increase the numerical precision of the gradients.

Question 9

What is the primary motivation for using Position Embeddings in a transformer model?

Accepted Answer

To enable the self-attention mechanism to recognize the order of tokens.. Because the self-attention mechanism is permutation-invariant, it treats every token as if it were independent of its position. Without explicit position information, a transformer would struggle to understand syntax and order-dependent relationships. Adding position embeddings to input tokens injects this necessary structural information, allowing the model to distinguish between 'The dog bit the man' and 'The man bit the dog,' which is vital for language understanding.

Answer

To reduce the computational burden of attention calculations.

Answer

To optimize the model for inference on NVIDIA Jetson devices.

Answer

To perform dimensionality reduction on input vocabulary.

Question 10

Which THREE techniques are commonly used to improve the efficiency of inference for large language models?

Accepted Answer

Weight Quantization to reduce the bit-width of model parameters.. Inference efficiency is critical for deploying LLMs in real-world production environments. Techniques like quantization reduce model size and memory requirements, KV-caching prevents redundant computation of attention keys and values, and pruning eliminates unnecessary weights. These methods combined allow models to run with significantly lower latency and reduced hardware costs on NVIDIA inference hardware, making high-performance AI more accessible and scalable for diverse enterprise applications.

Answer

Increasing the number of transformer layers to add depth.

Answer

Replacing all activations with Sigmoid functions for speed.

Question 11

What is the role of 'Temperature' in the context of LLM text generation?

Accepted Answer

It adjusts the probability distribution before sampling the next token.. Temperature controls the randomness of the model's output distribution. A low temperature makes the model more confident and deterministic by sharpening the probability distribution, favoring the most likely next token. Conversely, a high temperature flattens the distribution, increasing the likelihood of selecting less probable tokens. This parameter is crucial for balancing creativity and coherence in generative AI applications, allowing users to fine-tune the model's output behavior.

Answer

It regulates the cooling of the GPU hardware.

Answer

It determines the maximum length of the generated sequence.

Answer

It compresses the model weights to save memory.

Question 12

Which component of an NVIDIA AI stack is primarily responsible for providing a low-level API for high-performance collective communication primitives across multi-GPU nodes?

Accepted Answer

NCCL. NCCL (NVIDIA Collective Communications Library) is the industry standard for multi-GPU communication. It is designed to provide high-performance primitives such as All-Reduce, All-Gather, and Broadcast, which are essential for distributed training and inference. Understanding NCCL is fundamental for scaling models across nodes, as it directly impacts the efficiency of gradient synchronization and model parallel strategies in high-performance computing environments.

Answer

cuBLAS

Answer

cuDNN

Answer

TensorRT

Question 13

A researcher is fine-tuning a large language model using PEFT (Parameter-Efficient Fine-Tuning) techniques. Which method is specifically designed to inject trainable low-rank matrices into the transformer layers to reduce the number of trainable parameters?

Accepted Answer

LoRA. Low-Rank Adaptation (LoRA) is the standard PEFT method for injecting trainable matrices into frozen pre-trained models. By minimizing the number of updated parameters, it significantly reduces VRAM requirements and storage overhead during the fine-tuning process. This technique is vital for deploying custom LLMs on resource-constrained hardware, as it maintains model performance while drastically simplifying the memory demands of the training pipeline.

Answer

Prefix Tuning

Answer

Prompt Tuning

Answer

Full Fine-Tuning

Question 14

Refer to the exhibit. The configuration shows the use of FSDP with mixed precision. What is the main benefit of using 'bf16' (Bfloat16) over 'fp16' in this context?

Accepted Answer

It provides a larger dynamic range, preventing gradient underflow.. Bfloat16 provides the same dynamic range as FP32, preventing the underflow issues common with FP16 when calculating gradients during deep learning training. This stability allows for training without loss-scaling techniques, simplifying the pipeline and improving convergence consistency. In high-performance training, using BF16 is the standard for modern GPUs like the NVIDIA A100/H100, as it ensures stability without sacrificing speed or requiring complex hyperparameter tuning for numeric precision.

Answer

It reduces the size of the model weights on the disk.

Answer

It is natively supported on older Pascal-based architectures.

Answer

It doubles the throughput compared to FP32.

Question 15

When evaluating an LLM for factual accuracy, which metric is most effective at detecting hallucinations compared to simple word-overlap metrics?

Accepted Answer

NLI-based entailment scoring. Hallucinations in LLMs are semantic errors that word-overlap metrics like BLEU or ROUGE fail to capture, as they only measure token matching. Model-based evaluation, using frameworks like RAGAS or NLI (Natural Language Inference) models, assesses the truthfulness of generated content against reference contexts. This is critical for building trustworthy enterprise AI, where factual accuracy is paramount and simple text similarity does not suffice for assessing the validity of generated information.

Answer

BLEU Score

Answer

ROUGE-L

Answer

Perplexity