NCP-GENL Practice Questions 2 — 50 Questions

Question 1

Refer to the exhibit. Which adjustment is the most immediate and effective way to resolve this OOM error while maintaining the same training architecture?

Accepted Answer

Decrease the batch size per GPU.. The error indicates that the current batch size is too large for the available VRAM. Reducing the batch size is the most direct solution to free up enough memory for the model to continue training. If a larger effective batch size is required for convergence, the user can subsequently enable gradient accumulation to compensate for the reduced per-step batch size without increasing the memory footprint.

Answer

Increase the sequence length of the inputs.

Answer

Switch from mixed precision to full FP32 precision.

Answer

Disable the optimizer state checkpointing.

Question 2

Which optimization method should be prioritized when the model's inference performance is bottlenecked by the CPU-to-GPU data transfer overhead?

Accepted Answer

GPU-based Pre-processing. When data transfer between the host and GPU is the bottleneck, the most effective strategy is to move the pre-processing logic onto the GPU itself. Using CUDA-accelerated kernels to perform operations like tokenization or normalization directly on the GPU avoids moving data across the PCIe bus, which is a slow operation that stalls the GPU during the inference pipeline.

Answer

Quantization

Answer

Model Pruning

Answer

Operator Fusion

Question 3

A team is fine-tuning a 13B-parameter Llama model with NVIDIA NeMo Framework on a single A100 80GB GPU. They apply LoRA adapters to the attention projection layers, but the adapters are producing negligible changes to model behavior even after several epochs, and the loss curve stays flat. They confirm the dataset is clean and the tokenizer is correct. Which LoRA configuration issue is the most likely cause?

Accepted Answer

The LoRA alpha scaling value is set extremely low relative to the rank, so the adapter's effective update magnitude is near zero.. LoRA updates are scaled by alpha divided by rank, so an alpha that is very small relative to rank shrinks the adapter's effective contribution and can leave the loss nearly unchanged. Correcting the alpha-to-rank ratio restores meaningful adapter influence, allowing the fine-tune to actually shift model behavior on the target task.

Answer

The optimizer was configured with a momentum value of zero, preventing the adapter weights from accumulating updates.

Answer

The LoRA target modules were set to the embedding layer rather than the attention projection layers.

Answer

The base model weights were accidentally left trainable, so gradients are flowing into the full model instead of the adapters.

Question 4

An enterprise deploying a large language model on an NVIDIA A100 GPU experiences high memory bandwidth bottlenecks during autoregressive token generation. Which optimization technique specifically addresses this memory-bound phase by merging element-wise operations and reducing global memory round-trips?

Accepted Answer

Enabling TensorRT-LLM custom kernel fusion for multi-head attention and activation blocks.. Kernel fusion combines multiple successive GPU operations, such as bias additions and activations, into a single CUDA kernel. This drastically reduces high-latency global memory read and write operations, directly mitigating the memory bandwidth bottleneck characteristic of autoregressive transformer decoding phases on NVIDIA hardware.

Answer

Applying static INT8 post-training quantization to all linear layers.

Answer

Increasing the global batch size to maximize arithmetic intensity.

Answer

Switching from FlashAttention to standard vanilla self-attention mechanisms.

Question 5

Which hardware component of an NVIDIA GPU is most responsible for accelerating matrix-multiply-accumulate (MMA) operations used in transformer layers?

Accepted Answer

Tensor Cores.. Tensor Cores are specialized hardware units designed to perform high-speed matrix-multiply-accumulate operations in a single clock cycle. By accelerating these core operations, Tensor Cores provide the massive compute throughput needed for deep learning. Understanding the role of Tensor Cores is vital because they define the performance limits for modern LLMs, and optimizing code to utilize them is the single most important task in GPU performance tuning.

Answer

CUDA Cores.

Answer

Streaming Multiprocessor (SM) Scheduler.

Answer

L2 Cache controller.

Question 6

Which prompt engineering strategy helps the model maintain focus when processing an extremely long document within a single context window?

Accepted Answer

Instruct the model to analyze the document in sections and summarize each.. Long context windows can lead to the 'Lost in the Middle' phenomenon, where the model performs better on the beginning and end of the document but ignores the middle. Using 'attention-focusing' instructions or prompting the model to summarize segments before synthesizing a final answer helps maintain performance across the entire document. This is critical for NVIDIA engineers working with large-scale technical whitepapers and documentation.

Answer

Always increase the model's temperature to 1.0.

Answer

Remove all system-level instructions to save tokens.

Answer

Limit the context window size to 1024 tokens.

Question 7

An enterprise is fine-tuning a large language model using NVIDIA NeMo Framework and encounters GPU out-of-memory errors during the backward pass. The training configuration already uses mixed-precision training (FP16). Which architectural intervention should be applied to resolve memory pressure while retaining the optimizer state precision?

Accepted Answer

Implement activation checkpointing to recompute intermediate activations during backward passes.. Applying activation checkpointing trades off compute for memory by recalculating activations during the backward pass instead of storing them all. This directly mitigates out-of-memory errors on NVIDIA GPUs during LLM fine-tuning without requiring a reduction in batch size or model accuracy, making it a standard best practice in NeMo Framework workflows.

Answer

Switch the global precision setting from FP16 to INT8 quantization.

Answer

Disable gradient accumulation completely to process each micro-batch independently.

Answer

Migrate the optimizer states from FP32 to FP8 format.

Question 8

Refer to the exhibit. What is the most likely cause of the failure based on the log entries?

Accepted Answer

The KV cache size is exceeding the allocated GPU memory limit.. The logs indicate high processing latency followed by KV cache eviction and finally an OOM error. This sequence suggests that the system is running out of VRAM due to the growing KV cache during long-context generation. As the context length increases, the memory required for the KV cache exceeds the available capacity, forcing evictions, slowing down processing, and ultimately triggering an Out of Memory crash.

Answer

The model weights are corrupted in the repository.

Answer

Network latency is causing the client-side timeout.

Answer

The GPU driver version is incompatible with TensorRT.

Question 9

In the context of LLM deployment, why is it recommended to use a dedicated inference server like Triton rather than a basic Flask or FastAPI wrapper?

Accepted Answer

Triton manages concurrent model execution and hardware utilization.. While basic web frameworks are easy to implement, they lack the sophisticated features required for production AI. Triton offers hardware-optimized concurrency, automatic dynamic batching, support for multiple backends (TensorRT, PyTorch, ONNX), and advanced metrics. These features are critical for handling the high computational demands and low-latency requirements of LLMs, providing better scalability and reliability compared to custom-built wrappers that do not natively manage GPU streams and queues.

Answer

Flask is incompatible with the NVIDIA CUDA drivers.

Answer

Custom wrappers require more memory than Triton servers.

Answer

Triton is the only way to support REST API endpoints.

Question 10

What is the primary function of data 'normalization' in the context of preparing inputs for a Transformer model?

Accepted Answer

Standardizing the text to ensure consistent interpretation of tokens.. Normalization, such as standardizing whitespace, handling special tokens, and ensuring consistent character encodings, ensures that the model interprets input text in a predictable manner. By removing noise that doesn't contribute to semantic meaning, the model can focus its capacity on learning complex linguistic patterns. This is a standard and essential step in any high-performance AI data pipeline using NVIDIA accelerated computing.

Answer

Increasing the complexity of the input text to help the model learn more features.

Answer

Encrypting the dataset to protect sensitive information during training.

Answer

Compressing the dataset to reduce storage space on the GPU disk.

Question 11

When assessing the quality of a dataset for instruction fine-tuning, which TWO metrics or methods are considered most reliable for measuring dataset diversity?

Accepted Answer

Embedding-based clustering to visualize topical coverage across the corpus.. Dataset diversity ensures that the model encounters a broad range of topics and linguistic patterns, which is critical for generalization. Using embedding-based clustering allows for the identification of thematic coverage, while perplexity distribution analysis helps assess whether the dataset contains a balance of common and complex structures. These methods together provide a quantitative view of the data's breadth, reducing the risk of bias or overfitting.

Answer

Calculating the total number of words in the dataset.

Answer

Measuring the average response length for every instruction.

Answer

Checking if the dataset is solely in ASCII format.

Question 12

Why is it important to perform 'domain-specific' data cleaning when preparing a corpus for fine-tuning a medical LLM?

Accepted Answer

Generic cleaning often removes or modifies critical domain-specific nomenclature.. Medical text contains specific jargon, abbreviations, and relationships that generic cleaning might misinterpret. Generic tools often remove or alter terms that are critical to medical context, such as drug names or procedural codes. By using domain-specific cleaning, you ensure that the model retains the precise terminology necessary for high-stakes, accurate clinical reasoning in medical applications.

Answer

It helps to anonymize the patient data by replacing all medical terms with generic labels.

Answer

It ensures that the dataset size is significantly reduced to fit in GPU cache.

Answer

It automatically corrects all scientific inaccuracies present in the original documents.

Question 13

Which metric is most critical to monitor for identifying 'bottlenecks' in a high-throughput LLM deployment?

Accepted Answer

GPU Utilization percentage.. GPU utilization is the primary metric for understanding how well the model is saturating the compute resources. If GPU utilization is low while latency is high, it suggests a bottleneck elsewhere, such as CPU preprocessing, data transfer, or synchronization issues. Monitoring this metric allows engineers to determine if they need to increase batch sizes, optimize the pipeline, or scale the infrastructure to maintain performance.

Answer

Total number of files in the model repository.

Answer

The name of the backend framework used for training.

Answer

The number of times the server was restarted.

Question 14

An LLM application deployed on NVIDIA Triton Inference Server is experiencing intermittent latency spikes. Which metric is most critical to monitor to determine if the issue stems from GPU compute saturation?

Accepted Answer

SM (Streaming Multiprocessor) occupancy. Monitoring GPU utilization alone is insufficient because it does not distinguish between compute saturation and memory bandwidth bottlenecks. NVML-based metrics specifically tracking SM (Streaming Multiprocessor) occupancy provide the granular insight required to identify whether the execution cores are fully utilized. This metric is essential for capacity planning and ensuring that the inference throughput meets strict service level agreements under high concurrent request loads in production environments.

Answer

Total system CPU usage

Answer

Network interface throughput

Answer

Total GPU memory capacity

Question 15

A team is testing a new LLM application. During red-teaming, the model consistently leaks sensitive internal project codenames. How should the team address this systematically?

Accepted Answer

Deploy a guardrail that filters output against a list of sensitive terms.. Systematic mitigation of data leakage requires a multi-layered approach. Modifying the base model is rarely sufficient; instead, one must implement output-side guardrails that perform pattern matching and dictionary-based filtering. This ensures that even if the model attempts to generate sensitive info, the guardrail intercepts it. This process protects intellectual property and maintains compliance with corporate confidentiality agreements by ensuring that protected information remains strictly within the secure environment.

Answer

Increase the number of training epochs on the existing dataset.

Answer

Add a disclaimer at the end of every response stating that the content is confidential.

Answer

Randomize the model's weights during every inference run.

Question 16

An enterprise fine-tunes a Llama-3-70B model using NVIDIA NeMo for automated technical support ticketing. The development team needs an automated evaluation pipeline that measures semantic similarity against human-curated reference answers without relying on costly human annotators. Which metric provides the most robust embedding-based semantic similarity assessment for this scenario?

Accepted Answer

BERTScore computes token similarity matrices using contextual embeddings to capture deep semantic meaning regardless of surface-level phrasing variations.. BERTScore leverages contextual embeddings from transformer models to evaluate token-level semantic overlap rather than exact string matching, making it ideal for technical support text where phrasing varies. This automated metric correlates strongly with human judgment, significantly accelerating iteration cycles during enterprise model development workflows on NVIDIA infrastructure.

Answer

ROUGE-1 measures unigram overlap but fails to capture semantic synonyms and contextual nuances common in enterprise technical support documentation.

Answer

BLEU evaluates n-gram precision with a brevity penalty, heavily penalizing valid creative paraphrasing typically found in conversational AI support responses.

Answer

Perplexity measures how well a probability distribution predicts a sample, reflecting language fluency rather than semantic alignment with a specific reference answer.

Question 17

Which TWO of the following practices are recommended for ensuring ethical AI development when using NVIDIA NIMs in an enterprise environment?

Accepted Answer

Perform periodic bias audits on model responses using diverse, representative evaluation datasets.. Ensuring ethical AI involves both technical validation and process transparency. Regular bias auditing helps detect unintended stereotyping, while implementing robust logging ensures accountability for every generated output. These practices allow organizations to monitor for discriminatory patterns and maintain an audit trail for compliance, which is essential for building trust with users and adhering to global ethical standards for AI deployment in sensitive business sectors.

Answer

Bypass local logging to optimize inference latency for high-throughput applications.

Answer

Use the model in a closed-loop system where users cannot report offensive content.

Answer

Exclude all metadata from model outputs to prevent potential privacy leaks.

Question 18

An engineer is designing a Transformer-based model for long-context document summarization. They decide to replace the standard dense self-attention mechanism with a sliding window attention approach. What is the primary architectural implication of this change?

Accepted Answer

The computational complexity of the self-attention mechanism is reduced from quadratic to linear.. Sliding window attention restricts the receptive field of each token to a local neighborhood, drastically reducing the quadratic memory complexity of standard attention to linear. This is critical for scaling LLMs to long contexts, as it prevents the O(n²) memory growth that typically causes GPU out-of-memory errors on large input sequences while maintaining local coherence.

Answer

The model's total parameter count increases due to added positional embedding layers.

Answer

The model loses the ability to perform parallel training across multiple GPU nodes.

Answer

The model is no longer compatible with standard softmax normalization functions.

Question 19

An enterprise deployment of NeMo Guardrails is experiencing hallucinations where the model provides medical advice despite strict system prompts. What is the most effective approach to mitigate this risk?

Accepted Answer

Implement NeMo Guardrails 'dialogue rails' to detect and redirect queries related to medical diagnosis.. NeMo Guardrails provides a structured way to intercept model input and output to enforce safety boundaries. By defining specific 'rails' that detect non-compliant topics, the system can pivot the conversation or block the response entirely. This mechanism is critical in high-stakes environments where adherence to strict safety protocols is mandatory to prevent liability and ensure that generative AI tools do not cross into domains requiring human expertise.

Answer

Increase the temperature parameter of the LLM to provide more creative, diverse outputs.

Answer

Retrain the entire foundation model on a curated dataset of medical textbooks.

Answer

Change the model architecture to a smaller parameter size to reduce knowledge density.

Question 20

A team is pretraining a 13B-parameter decoder-only LLM on English text using byte-pair encoding with a 50,000-token vocabulary. They observe that the model produces fluent but repetitive continuations and that the average log-probability assigned to ground-truth tokens plateaus early. The training loss curve shows the model is underfitting rather than overfitting. Which architectural change is most likely to improve the model's capacity to capture long-range dependencies?

Accepted Answer

Increase the number of transformer layers and the hidden dimension proportionally.. Underfitting in a large decoder-only LLM indicates the model lacks sufficient capacity for the data distribution. Scaling both depth and width increases the parameter count and the number of nonlinear transformations applied to the residual stream, which improves the model's ability to capture long-range dependencies. The other options either rebalance existing capacity, change positional encoding without adding parameters, or reduce computation without improving fit.

Answer

Reduce the vocabulary size to 32,000 tokens to lower the softmax computation cost.

Answer

Increase the number of attention heads while keeping the hidden dimension fixed.

Answer

Replace the learned positional embeddings with sinusoidal absolute positional encodings.