NCA-GENL Practice Test 11 — 15 Questions

Question 1

Which THREE of the following strategies are recommended by NVIDIA to mitigate data leakage in enterprise-grade LLM applications?

Accepted Answer

Applying PII masking techniques on raw data prior to the training phase.. Data leakage occurs when sensitive information is unintentionally included in the model's training set or emitted in its response. Mitigating this requires a defense-in-depth approach: sanitizing training data to remove PII, implementing strict access controls for users, and utilizing output guardrails to intercept sensitive patterns (like credit card numbers) before they reach the user. These measures collectively protect user privacy and corporate integrity, ensuring that sensitive data remains secure throughout the model lifecycle.

Answer

Sharing the entire training dataset publicly to increase model transparency.

Answer

Increasing the model parameter count to improve internal data encryption.

Question 2

An enterprise AI researcher is conducting ablation studies on a large language model using NVIDIA NeMo. To ensure the experimentation results are scientifically valid and statistically sound, which THREE practices must be enforced during the study?

Accepted Answer

Evaluate across multiple random initialization seeds to account for stochastic variance in training.. Rigorous ablation studies require isolating individual components while holding all other experimental variables constant. Enforcing multiple random seeds accounts for initialization variance, controlling confounding hyperparameters prevents attribution errors, and applying rigorous statistical testing confirms whether observed performance deltas are statistically significant.

Answer

Vary multiple architectural hyperparameters simultaneously in every single experimental iteration.

Answer

Discard all experimental runs that fail to meet performance expectations without logging the failure.

Question 3

You are refining a dataset for a domain-specific LLM using NVIDIA NeMo. You want to visualize the similarity of documents to ensure your training set covers the required technical domains effectively. Which tool and visualization combination is best suited for this?

Accepted Answer

UMAP projection of document embeddings. Using Sentence-BERT embeddings combined with UMAP (Uniform Manifold Approximation and Projection) is the standard for high-dimensional document visualization. UMAP preserves both local and global data structures better than other methods. By plotting these embeddings, you can visually confirm if your training data covers the entire technical scope required and identify any 'holes' or clusters of off-topic data that need to be removed or augmented.

Answer

K-means clustering represented by simple bar charts

Answer

A word cloud generated from the top 1000 terms

Answer

A heatmap of raw document-term matrices

Question 4

Refer to the exhibit. The model is failing with an OOM at layer 42 during training. What visualization would most likely point to the cause of the memory fragmentation?

Accepted Answer

A memory allocation timeline plot per layer. The OOM and high fragmentation are caused by the interaction of dense-attention mechanisms and large sequence lengths (32k). Dense attention scales quadratically with sequence length, consuming massive memory. A heatmap of 'memory allocation per layer' or a 'memory timeline plot' would reveal the memory spikes during the attention computation stage, confirming that the current architecture requires FlashAttention or sequence parallelization to manage memory more efficiently.

Answer

A histogram of training loss values

Answer

A 3D surface plot of GPU core clock speeds

Answer

A line chart showing model weight distribution

Question 5

Which THREE actions are recommended for establishing a robust 'Human-in-the-Loop' (HITL) system for an AI deployment?

Accepted Answer

Setting specific confidence thresholds that trigger human review.. An effective Human-in-the-Loop system balances AI speed with human judgment. The three critical steps are defining clear escalation triggers, providing tools for human intervention, and maintaining a feedback loop where human corrections improve the model. These actions ensure that humans remain the ultimate authority in high-stakes scenarios, directly supporting the Trustworthy AI goals of oversight, accountability, and continuous improvement through expert guidance.

Answer

Automating all processes to eliminate the potential for human error.

Answer

Restricting human access to the model's internal weights and architecture.

Question 6

You are running an experiment comparing two different fine-tuning methods (Full Fine-tuning vs. PEFT). Why is it crucial to keep the dataset and evaluation benchmark identical?

Accepted Answer

To ensure that performance differences reflect the tuning method.. Scientific rigor demands that all variables except the independent variable be controlled. If the dataset or evaluation criteria differ, it becomes impossible to determine if the performance variance is caused by the fine-tuning method or by the inherent differences in the evaluation content. Consistent evaluation acts as a 'control' for the experiment, ensuring that the findings regarding the effectiveness of the fine-tuning technique are statistically valid and replicable.

Answer

To minimize the computational cost of the experiment.

Answer

To prevent the model from crashing due to memory issues.

Answer

To speed up the training time of the PEFT approach.

Question 7

When evaluating LLM output quality using human-in-the-loop data, which THREE metrics or techniques are most effective for detecting systemic hallucinations?

Accepted Answer

Consistency check across multiple temperature settings.. Detecting systemic hallucinations requires a combination of automated consistency checks and structured human evaluation. Consistency across multiple temperature settings, NLI (Natural Language Inference) against ground truth, and human-labeled factuality scores provide a robust framework. By triangulating these metrics, developers can quantify the frequency and severity of model fabrications, which is critical for safety and reliability in generative AI deployment, ensuring users receive accurate and trustworthy information rather than confident but incorrect model responses.

Answer

Word count distribution analysis.

Answer

Model training loss convergence tracking.

Question 8

During the evaluation of a Large Language Model, you notice that the model consistently predicts the most frequent tokens regardless of the context. Which visualization would most clearly illustrate this phenomenon of 'probability collapse'?

Accepted Answer

A histogram of token probability distributions. Probability collapse occurs when the model's output distribution becomes overly concentrated on a few high-probability tokens, ignoring the diversity of the context. A probability distribution histogram of the model's top-k predictions shows a sharp peak at the most likely token, with near-zero probability for others. Visualizing this for various prompts demonstrates the lack of entropy, signaling that the model is failing to utilize its full vocabulary effectively.

Answer

A scatter plot of average response length

Answer

A line chart of training loss over time

Answer

A bar chart showing total inference time

Question 9

What is the primary purpose of using a Model Repository in the NVIDIA Triton Inference Server architecture?

Accepted Answer

To manage model versioning and configuration dynamically.. The Model Repository is a centralized, organized directory structure that Triton monitors to load and serve models. It provides a standardized interface for managing model versions, configurations, and dependencies. This structure is critical for version control, allowing developers to roll back models, conduct A/B testing, and ensure consistent deployment across multiple environments, which is essential for maintaining system stability and reliability in production software development.

Answer

To store raw training data for future fine-tuning.

Answer

To act as a high-performance vector database.

Answer

To serve as a GUI for end-user model interactions.

Question 10

When fine-tuning a model on a new dataset, why is it important to keep a portion of the original pre-training data in the fine-tuning mix?

Accepted Answer

To prevent catastrophic forgetting. Retaining pre-training data during fine-tuning prevents 'catastrophic forgetting,' where the model loses its general knowledge while adapting to new tasks. This practice is essential for maintaining the model's capabilities in reasoning, coding, or language fluency. For NVIDIA NCA-GENL standards, understanding how to preserve base model utility while specializing for specific domains is a critical skill for successful model lifecycle management.

Answer

To increase the total number of training epochs

Answer

To reduce the computational time of fine-tuning

Answer

To improve the hardware utilization rates

Question 11

Which visualization tool is most suitable for tracking the gradient norm evolution during the training of a large language model to detect vanishing or exploding gradients?

Accepted Answer

Line chart.. Line charts are the optimal choice for monitoring scalar values like gradient norms over time or iteration steps. By plotting the norm, data scientists can instantly recognize when gradients become excessively large or vanish, which indicates instability. Detecting these anomalies early is essential for adjusting hyperparameter settings such as learning rates or gradient clipping, ensuring the training process remains stable and the model achieves optimal convergence without stalling or diverging mid-training.

Answer

Scatter plot matrix.

Answer

Heat map.

Answer

Pie chart.

Question 12

Refer to the exhibit. Which technique is most effective for preventing the reported NaN error during model training?

Accepted Answer

Gradient Clipping. The exhibit shows a classic case of exploding gradients, where the gradient norm spikes and leads to non-finite weight values. Gradient clipping is the standard industry practice to mitigate this, as it caps the gradient magnitude during backpropagation. Implementing this ensures stability in deep networks, preventing training collapses that waste expensive GPU compute resources and time in large-scale AI development cycles.

Answer

Using a larger batch size

Answer

Enabling Layer Normalization

Answer

Changing the optimizer to SGD

Question 13

A team is pretraining a large language model on a cluster of NVIDIA GPUs. They observe that the model's training loss decreases steadily for the first few epochs but then suddenly spikes and eventually becomes NaN. They suspect this is due to exploding gradients. Which technique is most appropriate to address this issue?

Accepted Answer

Apply gradient clipping to limit the magnitude of gradients during backpropagation.. Exploding gradients cause large parameter updates that can destabilize training, leading to loss spikes and NaN values. Gradient clipping is a standard technique that caps gradient norms or values before the optimizer step, preventing such destructive updates. The other options either exacerbate the problem or do not directly target the root cause, making gradient clipping the most appropriate remedy for this scenario.

Answer

Reduce the batch size to decrease the variance of gradient estimates.

Answer

Increase the learning rate to help the model escape the spiking region.

Answer

Switch from Adam to stochastic gradient descent (SGD) without momentum.

Question 14

A data science team is running a controlled experiment with NVIDIA NeMo to compare two fine-tuning recipes for a 7B-parameter LLM: one with a constant learning rate and one with a cosine decay schedule. They notice the evaluation loss curves diverge significantly after step 500, but they cannot tell whether the difference is caused by the learning-rate schedule or by random seed variance. Which experimental change should they make to isolate the effect of the schedule?

Accepted Answer

Run both recipes with a fixed random seed and identical data ordering, then compare the loss curves.. A controlled experiment requires isolating the independent variable—here the learning-rate schedule—while holding all other factors constant. Random seed and data ordering are common sources of run-to-run variance in LLM fine-tuning. Fixing them across both recipes ensures that observed differences in evaluation loss are caused by the schedule rather than initialization or batch order, making the comparison valid.

Answer

Switch both recipes to a warmup-stable-decay schedule and compare the final evaluation loss instead of the curves.

Answer

Increase the batch size for both recipes until the loss curves become smoother and easier to compare visually.

Answer

Run each recipe on a different GPU type to see whether hardware differences explain the divergence.

Question 15

An ML engineer is setting up an experiment log for a fine-tuning run and wants to record the metadata necessary to reproduce the resulting model later. Which two items are most essential to capture for reproducibility? (Choose two.)

Accepted Answer

The random seed used for data shuffling and initialization.. Reproducibility requires capturing the inputs that determine the trained weights: the randomness source and the exact training data. The seed governs shuffling and initialization, and the dataset version pins the examples used. Together they let another engineer rerun the same configuration and obtain a comparable model, which is the practical definition of a reproducible experiment.

Answer

The names of the engineers who reviewed the training logs.

Answer

The wall-clock duration of each training epoch.

Answer

The GPU model and driver version used during training.