NCA-GENL Practice Test 4 — 25 Questions

Question 1

Which of the following is the primary goal of the 'Experimentation' phase in an LLM project?

Accepted Answer

Determining optimal configurations for a task. The experimentation phase focuses on hypothesis testing, hyperparameter tuning, and prompt engineering to find the optimal configuration for a specific task. By isolating variables, researchers can identify the best balance between accuracy, latency, and cost. This phase is essential for moving from a general-purpose model to a specialized, reliable solution, ensuring the project meets defined success criteria before proceeding to full-scale deployment.

Answer

Deploying the model to production

Answer

Purchasing new hardware for the team

Answer

Writing marketing copy for the product

Question 2

An organization is deploying an LLM for customer support. To ensure Trustworthy AI, which approach best mitigates the risk of model hallucination while maintaining factual grounding?

Accepted Answer

Implement Retrieval-Augmented Generation (RAG) using a vector database for source verification.. Retrieval-Augmented Generation (RAG) is the industry standard for grounding LLMs. By injecting validated, domain-specific context into the prompt, the model relies on provided documents rather than latent parameters. This architectural choice is critical for Trustworthy AI because it creates a verifiable audit trail, allowing the system to cite sources for its claims, which significantly reduces the probability of generating nonsensical or fabricated responses in customer-facing interactions.

Answer

Increase the model's temperature parameter to maximum to encourage diverse reasoning.

Answer

Apply fine-tuning on the entire customer support history to memorize expected responses.

Answer

Restrict the model to only use pre-computed templates for every possible customer query.

Question 3

A data scientist is preparing to perform hyperparameter tuning for a Retrieval-Augmented Generation (RAG) system. Which TWO parameters should be prioritized for experimentation to improve retrieval accuracy?

Accepted Answer

Chunk size. Retrieval accuracy in RAG systems is heavily influenced by the chunking strategy and the similarity search configuration. Experimenting with these two components allows developers to optimize the context provided to the LLM. Properly tuned retrieval ensures that the model has the most relevant information, which is the most critical factor in reducing hallucinations and improving the factual grounding of generated responses.

Answer

Model quantization bit-width

Answer

System prompt length

Answer

GPU clock speed

Question 4

You need to compare the performance of two different LLMs on a set of benchmark tasks. Which visualization technique is most appropriate for a side-by-side comparison of multiple performance metrics (e.g., accuracy, latency, and truthfulness)?

Accepted Answer

A radar chart showing normalized metrics. Radar charts (or spider plots) are ideal for comparing models across multiple distinct, normalized metrics. They allow for a comprehensive view of a model's strengths and weaknesses in a single plot. For instance, you can easily see if Model A excels in accuracy while Model B outperforms in latency, providing a clear visual basis for selecting the correct model for specific production use cases and requirements.

Answer

A standard line graph

Answer

A scatter plot with only two axes

Answer

A simple histogram of total parameter count

Question 5

Which of the following is a primary objective of 'Ablation Studies' in LLM experimentation?

Accepted Answer

To determine the contribution of individual components.. Ablation studies systematically remove components (like layers, heads, or data sources) to measure their specific contribution to model performance. This process is essential for understanding the model's architecture and optimizing it by removing redundant or inefficient parts. For NVIDIA-certified professionals, ablation studies provide the empirical evidence needed to defend design choices and justify model architectural simplifications in complex projects.

Answer

To increase the total number of parameters in the model.

Answer

To accelerate the training speed by using less data.

Answer

To debug and fix errors in the model's training code.

Question 6

Which of the following best describes the principle of 'Interpretability' in the context of Trustworthy AI?

Accepted Answer

The capability to explain the internal decision-making process in human-understandable terms.. Interpretability refers to the degree to which a human can understand the cause of a decision or prediction made by an AI model. In high-stakes domains like healthcare or finance, knowing 'why' a model arrived at a conclusion is as important as the conclusion itself. This transparency is crucial for building trust, debugging errors, and ensuring that the model complies with regulatory requirements regarding fairness and decision-making accountability.

Answer

The ability of the model to perform multiple tasks simultaneously without loss of accuracy.

Answer

The speed at which a model can process training data during the fine-tuning phase.

Answer

The process of removing all personal identifiers from the training dataset.

Question 7

Why is 'Data Provenance' considered a crucial component in maintaining Trustworthy AI?

Accepted Answer

It provides a clear audit trail for data lineage, ethics, and legal compliance.. Data provenance involves tracking the origin, history, and licensing status of the training data. For Trustworthy AI, it ensures legal compliance, intellectual property rights, and the ability to audit the training set for bias. Without a clear chain of custody for the data, organizations cannot guarantee that their models are trained on ethical, high-quality, and legally obtained information, leading to significant reputation and legal risks.

Answer

It ensures that the model can be compressed into a smaller size for edge deployment.

Answer

It speeds up the GPU training process by indexing the data in a vector database.

Answer

It automatically corrects grammatical errors in the training corpus.

Question 8

Which technique should an organization prioritize to identify and reduce systematic bias in a generative model's training dataset?

Accepted Answer

Conduct a comprehensive audit of the training corpus to identify demographic imbalances.. Bias mitigation must start at the data layer. Analyzing the dataset for representational imbalances, stereotype associations, and lack of diversity is the most effective way to address bias before model training begins. This process is essential for Trustworthy AI because it ensures that the foundational intelligence of the model is not built upon skewed or exclusionary data, which prevents the amplification of societal prejudices in downstream AI applications.

Answer

Apply post-processing filters to censor all sensitive keywords in model output.

Answer

Increase the model size to allow for better internal alignment with human values.

Answer

Use an adversarial model to guess the sensitive attributes of the primary model's output.

Question 9

When training a model with a very large dataset, which approach provides the best balance between computational efficiency and model convergence?

Accepted Answer

Mini-batch gradient descent.. Mini-batch gradient descent strikes an ideal balance by processing small subsets of the data (batches) at a time. This provides the efficiency of vectorization on GPU hardware while maintaining enough stochasticity to help the optimizer avoid poor local minima. Compared to full batch (too slow) or single-sample stochastic descent (too noisy), mini-batching is the industry standard for scaling training to large datasets while achieving optimal convergence.

Answer

Full-batch gradient descent.

Answer

Stochastic gradient descent with batch size 1.

Answer

Gradient descent without any data shuffling.

Question 10

Refer to the exhibit. This error occurs during the training of an LLM. What is the most likely cause for this 'device-side assert' error?

Accepted Answer

The target indices exceed the vocabulary size.. A device-side assert error in PyTorch usually indicates an index-out-of-bounds error or a shape mismatch during a kernel operation on the GPU. When the loss function expects a specific range of indices for targets (e.g., in a cross-entropy loss), providing values that exceed the vocab size triggers this assertion. This is a common debugging hurdle that highlights the importance of rigorous input data validation before submitting tensors to the GPU's highly optimized, non-interactive kernels.

Answer

The model has run out of VRAM for the current batch.

Answer

The GPU has overheated and shut down.

Answer

The learning rate is set too high for convergence.

Question 11

Which machine learning paradigm involves an agent learning to make decisions by performing actions in an environment to maximize a cumulative reward?

Accepted Answer

Reinforcement Learning. Reinforcement Learning (RL) is fundamentally about agents interacting with environments. Unlike supervised learning, which relies on labeled datasets, or unsupervised learning, which finds hidden structures, RL uses a feedback loop of rewards and penalties. This paradigm is crucial for robotics, game playing, and autonomous systems where the optimal sequence of actions is not pre-defined, but must be discovered through trial and error.

Answer

Supervised Learning

Answer

Unsupervised Learning

Answer

Transfer Learning

Question 12

Refer to the exhibit. Which strategy is most effective for resolving this memory error without changing the hardware?

Accepted Answer

Gradient Accumulation. The exhibit indicates that the GPU is nearly at capacity. Gradient accumulation is a standard technique that simulates a larger batch size by accumulating gradients over multiple smaller steps before performing a weight update. This allows the user to maintain the intended effective batch size while fitting the training within the physical memory limits of the GPU, ensuring the training process can proceed.

Answer

Increase the hidden layer size

Answer

Enable more data augmentation

Answer

Increase the number of training epochs

Question 13

When integrating an LLM into an application using NVIDIA API endpoints, what is the primary purpose of the 'System' role in the messages payload?

Accepted Answer

To define the model's behavioral guidelines and operational constraints.. The 'System' role provides foundational instructions that dictate the model's behavior, tone, and constraints throughout the conversation. It is a critical component in ensuring that the AI remains within the expected parameters of the application, such as maintaining a specific persona or following strict safety guidelines. Mastering this role is fundamental for developers to ensure consistent and high-quality outputs across different user interactions.

Answer

To provide the user with a history of previous successful interactions.

Answer

To act as a buffer for temporary memory storage during inference.

Answer

To specify the hardware architecture used for the inference request.

Question 14

Which THREE techniques are commonly used to improve the efficiency of inference for large language models?

Accepted Answer

Weight Quantization to reduce the bit-width of model parameters.. Inference efficiency is critical for deploying LLMs in real-world production environments. Techniques like quantization reduce model size and memory requirements, KV-caching prevents redundant computation of attention keys and values, and pruning eliminates unnecessary weights. These methods combined allow models to run with significantly lower latency and reduced hardware costs on NVIDIA inference hardware, making high-performance AI more accessible and scalable for diverse enterprise applications.

Answer

Increasing the number of transformer layers to add depth.

Answer

Replacing all activations with Sigmoid functions for speed.

Question 15

In the context of NVIDIA NeMo, which THREE actions are part of a robust experiment tracking workflow for fine-tuning?

Accepted Answer

Logging hyperparameters for every run. Robust experiment tracking is the foundation of reproducibility in machine learning. By logging configurations, monitoring metrics in real-time using tools like Weights & Biases or TensorBoard, and saving versioned checkpoints, researchers can compare results across iterations. These actions are vital for ensuring that performance gains are attributable to specific hyperparameter changes rather than random chance or environmental variations during training.

Answer

Deleting logs to conserve disk space

Answer

Manually calculating gradients during training

Question 16

Refer to the exhibit. Which technique is most appropriate to prevent this specific training failure?

Accepted Answer

Apply gradient clipping during the backpropagation step.. The exhibit shows a classic case of the exploding gradient problem, where large weight updates lead to numerical instability and NaN loss values. This is common in deep architectures, particularly RNNs or deep Transformers. Gradient clipping is the standard solution to constrain the norm of the gradients during backpropagation, ensuring that updates remain within a numerically stable range, thereby preventing the model parameters from reaching infinity or NaN states.

Answer

Increase the learning rate to bypass the threshold.

Answer

Remove the normalization layers to increase model capacity.

Answer

Change the optimizer from Adam to SGD without momentum.

Question 17

What is the primary benefit of tracking experiments using a centralized experiment management platform (e.g., Weights & Biases, MLflow)?

Accepted Answer

It ensures reproducibility by logging parameters and code state.. Centralized tracking provides a historical log of all runs, parameters, code versions, and results. This traceability is critical for reproducibility, allowing researchers to compare outcomes across months or even different team members. In an enterprise NVIDIA environment, this documentation prevents redundant experimentation and ensures that the best-performing models are easily identified and promoted for deployment into production.

Answer

It automatically scales GPU resources based on workload.

Answer

It enforces strict security policies on data access.

Answer

It increases the training speed of the LLM model.

Question 18

When deploying an LLM, what is the 'Time to First Token' (TTFT) metric used to measure?

Accepted Answer

The latency experienced before generation begins.. TTFT measures the delay between sending an inference request and receiving the first piece of generated text. This is a crucial metric for user experience because a long TTFT makes an application feel unresponsive. Developers must optimize model loading, initial compute, and network latency to keep this time low, ensuring that the generative response feels immediate and interactive for the end user during the application's runtime.

Answer

The total time taken to train the model.

Answer

The average number of tokens generated per second.

Answer

The amount of memory required to store the model.

Question 19

Refer to the exhibit. A developer wants to make the model's output more deterministic and focused on highly probable tokens. Which change should be made to the configuration policy?

Accepted Answer

Decrease temperature to 0.1 and top_p to 0.5.. Determinism in LLMs is controlled by the temperature and top-p sampling parameters. Lowering temperature reduces the randomness in the probability distribution, while lowering top-p restricts the sampling pool to a smaller subset of high-probability tokens. Adjusting these parameters is vital for applications requiring factual consistency and stable outputs, as they directly dictate the sampling strategy used during the token generation process in the inference engine.

Answer

Increase temperature to 1.5 and top_p to 1.0.

Answer

Set presence_penalty to 1.0.

Answer

Increase max_tokens to 4096.

Question 20

You are performing a bias audit on a fine-tuned chat model. You need to visualize the model's responses to sensitive prompts across various demographic categories. Which visualization is most effective for identifying systemic bias?

Accepted Answer

Grouped violin plots of toxicity scores by demographic. Grouped box plots or violin plots of sentiment or 'toxicity' scores are the standard for bias audits. By plotting these scores for different demographic categories (e.g., gender, ethnicity), you can easily identify statistical shifts in model response patterns. These visualizations make it clear if the model is behaving differently toward specific groups, which is critical for meeting ethical safety standards in production-ready generative AI.

Answer

A scatter plot of token generation speed

Answer

A simple bar chart of total word counts

Answer

A matrix plot of model layer activations