NCA-GENL Practice Questions 1 — 50 Questions

Question 1

During an experiment, the researcher decides to increase the model's sequence length. What is the most significant side effect they must manage?

Accepted Answer

The memory requirement for the attention matrix will grow quadratically.. Increasing sequence length in transformers typically leads to a quadratic increase in memory usage for the attention mechanism. This necessitates strategies like FlashAttention or model parallelism to prevent OOM errors. Understanding this trade-off is fundamental to the experimentation process, as it dictates the physical constraints and architectural choices available when building models that handle longer inputs for complex reasoning tasks.

Answer

The model will automatically converge in fewer training steps.

Answer

The learning rate must be increased to maintain stability.

Answer

The vocabulary size must be increased to accommodate new tokens.

Question 2

When integrating an LLM into a production application, you must protect against prompt injection. Which software engineering pattern is most effective for this purpose?

Accepted Answer

Implementing an asynchronous guardrail input filter.. Using a guardrail pattern, such as NVIDIA NeMo Guardrails, allows developers to intercept inputs and outputs to validate them against safety policies. This protects the LLM from malicious prompts and prevents the generation of harmful content. Guardrails are fundamental in enterprise AI software development because they provide a programmatic layer of control that enforces safety, compliance, and reliability, regardless of the underlying LLM's inherent behavior.

Answer

Hard-coding all possible malicious user inputs.

Answer

Increasing the model's temperature parameter.

Answer

Caching all prompts in a local database.

Question 3

Which THREE factors should a developer consider when choosing between FP16 and INT8 quantization for a production LLM deployment?

Accepted Answer

Hardware support for specific tensor core operations.. Choosing the correct precision balance is critical for optimizing LLM performance. FP16 offers a good compromise between quality and speed, while INT8 provides significant memory savings and increased throughput at the cost of potential precision loss. Understanding these trade-offs allows developers to align their model deployment with hardware constraints and accuracy requirements, ensuring the application maintains acceptable quality while meeting performance targets for end-user response times.

Answer

The availability of sufficient cooling for the server.

Answer

The compatibility with standard CSV file formats.

Question 4

A team fine-tuning a NeMo large language model runs the same training configuration three times and obtains validation loss values of 2.14, 2.31, and 2.09 at the end of the same number of steps. They need subsequent runs to produce tightly clustered, comparable numbers so hyperparameter comparisons are meaningful. Which change most directly addresses this problem?

Accepted Answer

Set a fixed random seed and enable deterministic behavior for the training run.. Run-to-run variance in loss comes from uncontrolled randomness: weight initialization, dropout sampling, data shuffling, and nondeterministic GPU kernels. Pinning a seed and enabling deterministic execution makes those sources identical across repeated runs, so differences in outcomes can be attributed to the hyperparameters under test rather than to chance.

Answer

Add early stopping based on validation loss plateau detection.

Answer

Increase the global batch size while holding the learning rate constant.

Answer

Switch the optimizer from AdamW to SGD with momentum.

Question 5

When evaluating LLMs for bias, what is the primary purpose of conducting a 'red teaming' exercise?

Accepted Answer

To identify vulnerabilities and edge cases that could lead to biased or harmful output.. Red teaming is a deliberate effort to stress-test the model by attempting to force it to output harmful, biased, or restricted content. By simulating adversarial attacks, developers can uncover latent vulnerabilities and systemic biases that standard testing might miss. This proactive evaluation is essential for building trustworthy systems, as it allows developers to implement necessary guardrails and safety filters before the model is deployed to production, thereby minimizing real-world harm.

Answer

To increase the speed of model inference in production environments.

Answer

To automate the generation of training data for fine-tuning the model.

Answer

To reduce the number of tokens required for long-form generation tasks.

Question 6

An enterprise machine learning team is training a large-scale transformer model on a cluster of NVIDIA A100 GPUs using mixed precision (FP16). During the initial training phase, the team notices sudden numerical underflow resulting in vanishing gradients and stalled loss convergence. Which optimization technique must be applied to mitigate this issue without sacrificing the memory-efficiency benefits of FP16?

Accepted Answer

Applying dynamic loss scaling to the loss value prior to backpropagation to shift gradients into the representational range.. Loss scaling multiplies the forward pass loss by a scaling factor to shift small gradient magnitudes into the representational range of the FP16 format, preventing underflow. This is essential in NVIDIA mixed-precision training because FP16 has a narrow dynamic range compared to FP32. Proper scaling prevents gradient values from truncating to zero while retaining half-precision throughput and memory footprint advantages on Tensor Cores.

Answer

Implementing gradient clipping by global norm to bound large gradient updates before parameter application.

Answer

Switching the entire training cluster to FP32 precision to guarantee maximum numerical stability and range.

Answer

Increasing the mini-batch size exponentially to smooth out noisy gradient estimates across distributed nodes.

Question 7

Refer to the exhibit. Which concept of Trustworthy AI is primarily demonstrated by the actions shown in the CLI output?

Accepted Answer

Safety and Robustness.. The logs demonstrate 'Safety and Robustness' through active content moderation. By identifying a toxicity score above the acceptable threshold and programmatically blocking the response, the system prevents the dissemination of harmful content. This is a core requirement of Trustworthy AI, ensuring that models operate within defined safety guardrails and actively mitigate the risk of harmful output generation, even when triggered by user input.

Answer

Model Explainability.

Answer

Data Privacy.

Answer

Algorithmic Efficiency.

Question 8

When implementing Retrieval-Augmented Generation (RAG), why is the choice of 'Chunk Size' critical for model retrieval performance?

Accepted Answer

It determines the semantic density and context of retrieved segments.. Chunk size determines how much context is included in a single document segment during the retrieval process. If chunks are too small, the model lacks sufficient context to answer complex queries. If they are too large, the retrieval results contain excessive noise, causing the model to lose focus. Optimizing this balance is a core task in RAG engineering to ensure that the retrieved information is both relevant and comprehensive enough for the model to generate accurate responses.

Answer

It directly limits the number of documents in the vector database.

Answer

It affects the latency of the embedding model's inference.

Answer

It is solely determined by the GPU's memory capacity.

Question 9

A developer is optimizing a retrieval-augmented generation (RAG) pipeline using NVIDIA TensorRT-LLM. They notice excessive latency during the document retrieval phase before the generation starts. Which optimization strategy is most effective for this bottleneck?

Accepted Answer

Implement a GPU-accelerated vector database for similarity search.. Latency in RAG pipelines often stems from inefficient embedding lookups or serial processing. Moving vector search to a GPU-accelerated database or utilizing a cross-encoder for re-ranking ensures the model receives highly relevant chunks. This approach balances retrieval precision with speed, ensuring the LLM receives context without stalling the inference server, which is critical for real-time generative applications.

Answer

Increase the number of transformer layers in the LLM.

Answer

Use float64 precision for all vector embedding calculations.

Answer

Decrease the context window size of the retrieval model.

Question 10

Refer to the exhibit. How should a data scientist interpret this evaluation result regarding the recent model update?

Accepted Answer

The update has caused a statistically significant performance drop.. The score delta is negative, and the confidence interval does not overlap zero, meaning the performance regression is statistically significant. In the context of LLM deployment, this is a clear 'red flag' suggesting that the update has degraded the model's reasoning capabilities on the MMLU benchmark. Instead of deploying, the team must investigate the cause, such as data contamination or poor fine-tuning data, to prevent releasing a model that performs worse than the current production baseline.

Answer

The performance change is statistically insignificant.

Answer

The update has significantly improved model accuracy.

Answer

The results are inconclusive due to the small sample size.

Question 11

What is the primary function of the 'Attention' mechanism in Transformer models?

Accepted Answer

To enable the model to weigh the importance of different input tokens. The attention mechanism allows the model to compute weights that signify the relevance of different parts of the input sequence to one another, regardless of their distance. This captures long-range dependencies effectively, which traditional RNNs struggle with due to vanishing gradients over long sequences. Mastering this concept is key to understanding why transformers have become the dominant architecture for nearly all state-of-the-art generative AI and natural language processing applications.

Answer

To reduce the number of parameters in the model

Answer

To convert text into numerical vectors

Answer

To perform sequential processing of inputs

Question 12

What is the role of 'Temperature' in the context of LLM text generation?

Accepted Answer

It adjusts the probability distribution before sampling the next token.. Temperature controls the randomness of the model's output distribution. A low temperature makes the model more confident and deterministic by sharpening the probability distribution, favoring the most likely next token. Conversely, a high temperature flattens the distribution, increasing the likelihood of selecting less probable tokens. This parameter is crucial for balancing creativity and coherence in generative AI applications, allowing users to fine-tune the model's output behavior.

Answer

It regulates the cooling of the GPU hardware.

Answer

It determines the maximum length of the generated sequence.

Answer

It compresses the model weights to save memory.

Question 13

An organization is concerned about 'Model Drift' affecting the trustworthiness of their customer-facing chatbot. What is the most effective way to monitor and address this issue?

Accepted Answer

Implement continuous evaluation metrics to detect performance degradation.. Model drift occurs when the model's performance degrades over time because the real-world environment changes, making the training data obsolete. Trustworthy AI requires continuous monitoring to detect these performance shifts. By establishing an evaluation pipeline that periodically tests the model against current benchmarks, organizations can identify drift early and trigger retraining, ensuring the system remains accurate, relevant, and reliable in the face of changing user behaviors.

Answer

Hard-code all possible responses to prevent the model from learning new patterns.

Answer

Increase the number of parameters in the model to improve its reasoning capacity.

Answer

Disable the model's feedback collection to avoid processing incorrect user data.

Question 14

You are performing a comparative analysis of two different LLM architectures by visualizing their performance on a RAG (Retrieval-Augmented Generation) benchmark. Which visualization is best for comparing the distributions of answer accuracy scores?

Accepted Answer

Box plot comparing accuracy scores.. Box plots (or box-and-whisker plots) provide a compact summary of data distribution, including median, quartiles, and outliers. When comparing two architectures, they allow for an immediate visual assessment of consistency, range, and bias. This is crucial in RAG benchmarking because high accuracy is insufficient; developers need models that consistently perform well across diverse queries, and box plots reveal whether one architecture suffers from more frequent low-quality outliers than the other.

Answer

Stacked bar chart of token counts.

Answer

Radial plot of training time.

Answer

Individual data point scatter plot.

Question 15

Refer to the exhibit. An engineer is tuning a deployment config. Why is 'enable_cuda_graph' set to true in this JSON configuration?

Accepted Answer

To reduce CPU overhead during repetitive kernel launches.. CUDA Graphs capture a sequence of GPU work as a single graph, reducing CPU overhead associated with kernel launches. This is critical for LLMs where many small kernel calls can lead to CPU-bound execution. In high-performance generative AI scenarios, reducing launch latency is essential to ensure that the GPU remains saturated with work, thereby maximizing tokens-per-second and reducing total request latency for end-users.

Answer

To increase the maximum batch size to 256.

Answer

To convert the model precision from fp16 to fp8.

Answer

To force the model to use the CPU for inference.

Question 16

A bank uses an NVIDIA NIM microservice to host an LLM for loan pre-screening. Before go-live, the risk team must confirm that the model's outputs are reproducible and that any change in behavior can be traced to a specific model version. Which deployment practice best satisfies this requirement?

Accepted Answer

Pin the NIM container to an immutable image digest and record the model name, digest, and inference parameters in a model registry entry for each release.. Reproducibility and traceability require freezing the exact serving artifact and recording it. An immutable image digest locks the weights, tokenizer, and runtime, while a registry entry maps each release to that digest and its inference parameters. Logging, parallelism, or higher temperature do not bind an output to a specific model version, so they cannot satisfy an audit that must reconstruct which model produced a decision.

Answer

Run the model on two GPUs in a tensor-parallel configuration so responses are identical across replicas.

Answer

Enable verbose prompt logging on the NIM endpoint and retain the logs for 30 days.

Answer

Increase the model's temperature setting so the risk team can observe a wider range of outputs before approving the release.

Question 17

What is the primary role of 'Loss Scaling' when training deep learning models in FP16 precision?

Accepted Answer

To prevent gradient underflow in FP16 training. Loss scaling is essential because FP16 has a narrower dynamic range than FP32. Small gradient values can underflow to zero, causing the model to stop learning. By scaling the loss up before backpropagation, the gradients are kept within the representable range of FP16, and then scaled back down during the weight update, ensuring stable and effective training while maintaining the speed advantages of half-precision compute.

Answer

To increase the training speed of the GPU

Answer

To reduce the required GPU memory footprint

Answer

To normalize the input data distribution

Question 18

A team wants to load and run an optimized quantized LLM entirely inside a Python application with minimal dependencies, using a single high-level API that handles engine building and generation. They are not deploying a network service. Which component of the NVIDIA software stack is designed for this use case?

Accepted Answer

NVIDIA TensorRT-LLM's Python LLM API, which wraps engine build and generation behind a high-level runtime interface.. When the goal is to run an optimized, quantized LLM inside a Python process without standing up a service, the TensorRT-LLM Python LLM API is the fit. It abstracts engine building and token generation behind a high-level interface, avoiding client-server overhead. Serving platforms such as Triton or NIM and preprocessing libraries such as DALI solve different problems and would add unnecessary infrastructure or miss the requirement.

Answer

NVIDIA NIM microservices, which package models as containerized endpoints with standardized inference APIs.

Answer

NVIDIA DALI, which provides a data loading and augmentation library primarily for computer vision preprocessing pipelines.

Answer

NVIDIA Triton Inference Server, which loads models from a model repository and exposes HTTP and gRPC endpoints.

Question 19

A researcher is fine-tuning a large language model using PEFT (Parameter-Efficient Fine-Tuning) techniques. Which method is specifically designed to inject trainable low-rank matrices into the transformer layers to reduce the number of trainable parameters?

Accepted Answer

LoRA. Low-Rank Adaptation (LoRA) is the standard PEFT method for injecting trainable matrices into frozen pre-trained models. By minimizing the number of updated parameters, it significantly reduces VRAM requirements and storage overhead during the fine-tuning process. This technique is vital for deploying custom LLMs on resource-constrained hardware, as it maintains model performance while drastically simplifying the memory demands of the training pipeline.

Answer

Prefix Tuning

Answer

Prompt Tuning

Answer

Full Fine-Tuning

Question 20

A team is fine-tuning an LLM and wants to detect whether individual training examples are causing unusually large gradient updates. They plan to visualize per-example gradient norms alongside other diagnostics. Which two visualizations are most appropriate for identifying these influential examples? (Choose two.)

Accepted Answer

A sorted bar chart of per-example gradient norms with examples ranked from largest to smallest. Detecting influential examples requires per-example gradient information. A sorted bar chart ranks examples so the largest norms stand out, while a scatter plot of gradient norm versus loss adds context about why those norms are large. Together they let the team isolate and inspect the examples driving unusually large updates.

Answer

A line chart of the moving average of total training loss across epochs

Answer

A pie chart showing the proportion of examples in each loss decile

Answer

A heatmap of the model's attention weights for a single randomly chosen example