NCA-GENL Practice Questions 2 — 50 Questions

Question 1

Refer to the exhibit. A developer implements this NVIDIA NeMo Guardrails configuration. A user submits a query about financial advice. What is the expected behavior of the LLM?

Accepted Answer

The guardrail blocks the request because finance is not in the whitelist.. The configured guardrail includes topic whitelisting, which restricts the model to only discussing 'tech' and 'science'. When the user submits a financial query, the system identifies the topic mismatch. Since the policy does not explicitly allow financial topics, the guardrail intercepts the input, preventing the model from generating a response. This proactive filtering is a key component of trustworthy AI, ensuring models operate within predefined domain boundaries.

Answer

The model generates a generic response about finance using its internal knowledge.

Answer

The system processes the request because the toxicity score is below 0.85.

Answer

The model prompts the user to clarify if the finance query is related to technology.

Question 2

When designing an experiment to evaluate the performance of an LLM on a downstream classification task, which THREE factors should be controlled to ensure the results are comparable across different model sizes?

Accepted Answer

Decoding strategy (temperature, top-p). To conduct valid cross-model comparisons, one must neutralize external variables. Inference-time settings like decoding parameters, the specific prompt template, and the evaluation dataset must remain constant. If these vary, the results reflect differences in the evaluation environment rather than the intrinsic capabilities of the models being tested, rendering the experimentation data inconclusive for determining which model size is truly optimal for the specific classification application.

Answer

The learning rate used during fine-tuning

Answer

The hardware GPU generation (e.g., A100 vs H100)

Question 3

A developer is writing a Python service that calls an NVIDIA-hosted NIM endpoint for a Llama model. The service must recover gracefully when the endpoint returns HTTP 429 responses during peak traffic, without dropping user requests. Which implementation approach best satisfies this requirement?

Accepted Answer

Implement exponential backoff with jitter and a bounded retry count, then surface a fallback error to the caller.. Rate-limit responses require the client to slow down and retry deliberately. Exponential backoff with jitter spreads retry attempts so multiple clients do not collide, and a bounded retry count keeps latency predictable. Because some requests may still fail after all retries, returning a controlled fallback error preserves a defined service contract for callers rather than hanging or crashing.

Answer

Retry the request in a tight loop immediately after each 429 response until it succeeds.

Answer

Increase the client's request timeout value so the 429 responses stop occurring.

Answer

Switch the endpoint URL to a different NIM model and continue sending the same request payload unchanged.

Question 4

A team is comparing two LLM fine-tuning runs on the same dataset. Run A used a cosine learning-rate schedule, and Run B used a constant learning rate. They plot validation loss versus training step for both runs on the same axes. Run A's curve is smooth, while Run B's curve shows a sharp upward spike around step 800 and then recovers. The team wants to determine whether the spike in Run B indicates a data-order artifact or a genuine optimization instability. Which additional visualization is most useful for that diagnosis?

Accepted Answer

A scatter plot of per-batch training loss for Run B colored by batch index modulo epoch length. Plotting per-batch training loss with color encoding the batch's position modulo the epoch length exposes whether the spike recurs at a fixed data position. A recurring pattern implies a data-order artifact, such as a hard shard or mislabeled examples, while a random pattern implies optimization instability. Summary charts and hardware metrics do not preserve the temporal and data-order information needed for this diagnosis.

Answer

A bar chart of final validation loss for Run A and Run B

Answer

A heatmap of attention weights for one validation example

Answer

A line chart of GPU utilization for Run A and Run B

Question 5

An AI engineer at a financial services company is running an LLM experimentation pipeline using NVIDIA NeMo. The primary objective is to evaluate how different tokenizers affect the accuracy of a named entity recognition (NER) task on financial documents. The engineer has already fixed the model architecture, the training dataset, and the hyperparameters. To ensure the experiment isolates the effect of the tokenizer, which action should the engineer take next?

Accepted Answer

Replace the tokenizer with a different one while keeping all other configurations unchanged, then compare NER accuracy.. To isolate the effect of the tokenizer, the engineer must change only the tokenizer while keeping all other factors constant. This controlled approach ensures that any observed difference in NER accuracy is due to tokenization rather than confounding variables like model architecture or dataset size. Such isolation is fundamental to valid experimentation and enables clear conclusions about tokenizer impact.

Answer

Increase the training dataset size to improve NER accuracy before changing the tokenizer.

Answer

Fine-tune the model on a general-domain corpus before evaluating on financial documents.

Answer

Retrain the model with the same tokenizer but different random seeds to measure variance.

Question 6

Which TWO factors should be considered when evaluating the cost-benefit of an LLM experimentation strategy?

Accepted Answer

Total GPU hours per experiment. Evaluating the cost-benefit of LLM experimentation requires balancing the financial cost of compute with the performance gains observed. In professional environments, experiments must be scoped to maximize meaningful insights while minimizing wasted GPU cycles. This involves prioritizing experiments that are likely to yield the highest impact on model quality relative to the resources consumed by the training runs.

Answer

The color scheme of the monitoring dashboard

Answer

The popularity of the LLM framework used

Answer

The number of research papers published by the team

Question 7

Which of the following describes the 'Warm-up' phase in the context of training deep neural networks?

Accepted Answer

Gradually increasing the learning rate at the start of training.. Warm-up is a technique where the learning rate starts at a very low value and is gradually increased over the first few hundred or thousand steps. This prevents the model from diverging early in the training process when weights are randomly initialized and gradients can be unstable. Proper warm-up is essential for successfully scaling the training of modern LLMs on large compute clusters, ensuring stable convergence from the very first iterations.

Answer

Pre-loading the model into the GPU cache.

Answer

A method to reduce training time by caching gradients.

Answer

Reducing the batch size to fit in memory.

Question 8

An application requires streaming responses from a deployed LLM. Which communication protocol is most suitable for minimizing latency and ensuring efficient data delivery in a real-time generative AI application?

Accepted Answer

gRPC with server-side streaming.. Streaming generative responses requires a protocol that supports persistent connections and low-overhead message framing. gRPC with server-side streaming is the industry standard for high-performance AI inference, as it facilitates efficient binary serialization and maintains low latency across the network. Using this protocol is critical for user-facing applications where perceived latency is directly tied to the speed at which text tokens appear on the screen.

Answer

HTTP/1.1 REST without streaming support.

Answer

Standard SMTP email protocols.

Answer

Synchronous SQL query polling.

Question 9

An ML engineer is deploying a Transformer-based inference service on an NVIDIA TensorRT-LLM runtime. To maximize inference throughput and reduce latency under heavy concurrent user traffic, the engineer needs to select the optimal decoding batching strategy. Which technique allows multiple incoming dynamic sequence requests to be batched together at the token level rather than waiting for entire sequences to complete?

Accepted Answer

Inflight batching (continuous batching) that schedules individual generation iterations at the token level across requests.. Inflight batching (also known as continuous batching) dynamically groups individual generation steps from different requests at each token generation iteration. This eliminates the idle padding overhead and waiting times associated with traditional static batching, maximizing GPU compute utilization in LLM serving.

Answer

Static request batching that groups fixed numbers of incoming inference prompts and pads them to equal sequence lengths.

Answer

Data-parallel replica sharding that duplicates the model across separate GPU devices to process independent streams.

Answer

Model parallelism that splits individual transformer weight matrices across multiple interconnected GPU devices.

Question 10

A developer is using TensorRT-LLM to build a chatbot and wants to reduce the memory footprint of the KV cache during inference. Which technique should they use?

Accepted Answer

Enable paged KV cache. Paged KV cache is a memory management technique in TensorRT-LLM that divides the KV cache into fixed-size blocks, allowing non-contiguous storage and dynamic allocation. This reduces fragmentation and enables more sequences to fit in memory. It is the recommended approach to minimize KV cache memory footprint during inference.

Answer

Disable quantization

Answer

Increase the beam width

Answer

Use a larger batch size

Question 11

A developer is integrating a NeMo Guardrails configuration into an existing chatbot. They need to ensure that the LLM does not generate content related to unauthorized financial advice. Which mechanism should they implement to achieve this programmatic constraint?

Accepted Answer

Utilize Colang to define input rails that intercept and block restricted topics.. NeMo Guardrails uses Colang to define flows and dialogue rails that intercept and inspect the interaction between the user and the LLM. By defining specific canonical forms and guardrail flows, developers can force the model to refuse prompts that trigger sensitive topics. This approach is essential for production-grade applications where ensuring model safety and regulatory compliance is mandatory for deploying generative AI safely within enterprise environments.

Answer

Increase the temperature parameter to 1.0 to improve model creativity.

Answer

Enable standard logging in the inference server to identify bad prompts.

Answer

Apply a secondary LLM to re-write every user prompt to be neutral.

Question 12

What is the primary motivation for using Position Embeddings in a transformer model?

Accepted Answer

To enable the self-attention mechanism to recognize the order of tokens.. Because the self-attention mechanism is permutation-invariant, it treats every token as if it were independent of its position. Without explicit position information, a transformer would struggle to understand syntax and order-dependent relationships. Adding position embeddings to input tokens injects this necessary structural information, allowing the model to distinguish between 'The dog bit the man' and 'The man bit the dog,' which is vital for language understanding.

Answer

To reduce the computational burden of attention calculations.

Answer

To optimize the model for inference on NVIDIA Jetson devices.

Answer

To perform dimensionality reduction on input vocabulary.

Question 13

A researcher is training a large language model and notices the training loss plateaus early while validation loss increases. What is the most likely cause, and which action should be taken?

Accepted Answer

The model is overfitting; apply dropout or weight decay.. The symptoms described clearly indicate overfitting, where the model captures noise in the training set rather than generalizing to unseen data. In the context of large language models, this is a critical challenge. Implementing regularization techniques such as weight decay or dropout helps constrain model complexity, forcing it to learn more robust features rather than memorizing specific patterns, thereby improving overall model performance and generalizability.

Answer

The model is underfitting; increase the number of hidden layers.

Answer

The learning rate is too low; increase it to accelerate convergence.

Answer

The dataset is too small; reduce the batch size.

Question 14

In the context of Large Language Models, what is the primary purpose of 'Attention mechanisms' as introduced in the Transformer architecture?

Accepted Answer

To compute dynamic weights representing the relevance of tokens.. Attention mechanisms allow models to weigh the significance of different tokens in an input sequence relative to one another, regardless of their distance. By computing relevance scores, the model can capture long-range dependencies effectively. This capability is crucial for understanding context, syntax, and semantics, which were historically difficult for sequential models like RNNs that struggled with information retention over long sequences during the training process.

Answer

To reduce the number of parameters in the model.

Answer

To process input sequences sequentially for better memory.

Answer

To enforce a fixed context window for all inputs.

Question 15

Refer to the exhibit. Which hyperparameter configuration in the provided JSON is directly responsible for preventing overfitting through weight penalty?

Accepted Answer

weight_decay. The JSON defines a standard configuration for model training. The 'weight_decay' key is the parameter that implements L2 regularization. By penalizing large weight values in the loss function, it discourages the model from relying too heavily on specific features, thereby preventing overfitting. This is a common and essential hyperparameter to tune when training deep neural networks to ensure good generalization on unseen test data, as shown by the provided value of 0.05.

Answer

dropout_rate

Answer

learning_rate_scheduler

Answer

enable_gradient_check

Question 16

Which component of an NVIDIA Transformer Engine is specifically designed to accelerate training on supported GPUs by dynamically adjusting precision?

Accepted Answer

Transformer Engine. The NVIDIA Transformer Engine uses FP8 precision and dynamic scaling to drastically accelerate training for transformer-based models. By utilizing hardware-level support for mixed-precision, it maximizes throughput while maintaining accuracy. Understanding how these hardware-software co-optimizations function is critical for modern LLM development, as they allow for training larger models in less time and with lower power usage compared to standard FP32 or mixed-precision training workflows.

Answer

Tensor Cores

Answer

CUDA Streams

Answer

NCCL (NVIDIA Collective Communications Library)

Question 17

Refer to the exhibit. A monitoring script outputs this JSON for an LLM inference service. What does the 'p99' metric represent in this context?

Accepted Answer

The latency of the slowest 1% of requests.. The p99 metric represents the 99th percentile of latency, meaning 99% of requests are processed in under 145.2 milliseconds. In LLM production, p99 is the critical industry standard for measuring tail latency, ensuring that even the slowest requests remain within acceptable bounds for user experience. Monitoring this metric is vital because it reveals transient performance bottlenecks that averages or medians hide, ensuring reliable service levels for real-time generative AI applications.

Answer

The average latency of all requests processed.

Answer

The median latency observed during the period.

Answer

The total throughput of the inference server.

Question 18

Which THREE of the following factors are critical when choosing a foundation model for an enterprise generative AI application?

Accepted Answer

Model licensing and data privacy policies. Selecting a foundation model involves balancing technical performance, operational costs, and security requirements. Enterprise readiness requires understanding data privacy policies, licensing, and the ability to fine-tune the model for domain-specific tasks. Failing to evaluate these factors can lead to significant downstream issues, including legal risks, poor performance, or unmanageable infrastructure costs, making this selection process one of the most important steps in an AI project lifecycle.

Answer

The number of hidden layers in the model

Answer

The specific programming language used to build the model

Question 19

You are performing exploratory data analysis on a massive dataset for an LLM training pipeline. You need to visualize the distribution of token frequencies in a corpus of 10 billion tokens. Which visualization technique is most effective for identifying long-tail patterns in power-law distributions typical of natural language data?

Accepted Answer

A log-log scale scatter plot of frequency versus rank. A log-log scale plot is the standard for analyzing power-law distributions. In LLM tokenization, the Zipfian distribution means a few tokens appear very frequently while most appear rarely. By plotting frequency versus rank on both logarithmic axes, you transform the curved power-law distribution into a linear relationship, making it significantly easier to identify outliers, detect artifacts in the tokenization process, and validate the model's expected vocabulary coverage.

Answer

A standard linear scale histogram

Answer

A pie chart showing top 20 token percentages

Answer

A box plot summarizing token length statistics

Question 20

When utilizing NVIDIA NIM for deployment, why is it recommended to use a containerized environment?

Accepted Answer

To ensure dependency consistency and portability.. Containerization provides a consistent runtime environment across development, testing, and production, which is crucial for managing the complex dependencies of AI stacks like CUDA, cuDNN, and TensorRT. This approach minimizes 'works on my machine' issues and ensures that the model performance remains identical regardless of the underlying host configuration, making it the industry standard for deploying high-performance generative AI models at scale.

Answer

To bypass the need for CUDA drivers on the host machine.

Answer

To improve the GPU's clock speed by optimizing kernel distribution.

Answer

To automatically optimize the model's weight distribution for multi-GPU setups.