NCP-GENL Hard Questions — 25 Difficult Questions

Question 1

When profiling an application with NVIDIA Nsight Systems, which TWO metrics are most critical to identify if an application is limited by the PCIe bus?

Accepted Answer

Host-to-Device (H2D) throughput.. PCIe bus saturation occurs when the transfer rate of data between the host and the GPU becomes a bottleneck. By monitoring the H2D (Host-to-Device) and D2H (Device-to-Host) transfer metrics, developers can see if data movement consumes a disproportionate amount of execution time. Identifying these spikes is crucial because offloading data to the GPU is often the slowest part of a pipeline, and minimizing these transfers is key to scaling high-performance AI.

Answer

GPU Register usage count.

Answer

Shared memory bank conflict count.

Answer

SM clock frequency.

Question 2

Refer to the exhibit. The Triton Inference Server is failing to load a model. The configuration shows two instances assigned to GPU 0. What is the most likely cause and the correct remediation?

Accepted Answer

The total VRAM required for two instances exceeds the GPU capacity; reduce instance count to 1.. The error indicates that the two instances of the model are collectively requesting more VRAM than is available on the physical GPU. By default, Triton attempts to allocate memory for every configured instance upon startup. The remediation requires either reducing the number of instances or implementing a memory-aware model partitioning strategy, such as using Model Analyzer to determine the safe memory footprint per instance before deployment.

Answer

The model count exceeds the number of available CUDA cores, requiring a driver update.

Answer

The model weights are corrupted, preventing the inference engine from initializing the memory space.

Answer

The batch size is set to zero in the configuration, preventing memory allocation.

Question 3

Refer to the exhibit. An engineer is fine-tuning an LLM using the provided configuration. What is the primary purpose of applying 'yarn' scaling in this architecture?

Accepted Answer

It enables the model to process sequences longer than the original pre-trained limit.. YaRN (Yet another RoPE extension) is designed to extend the context window of pre-trained LLMs beyond their original training length. By modifying the Rotary Positional Embeddings (RoPE) base frequency, it allows the model to interpolate position indices without severe degradation in perplexity. This is essential for enterprise deployments requiring document retrieval or analysis tasks where input lengths exceed the base model's pre-trained constraints.

Answer

It increases the number of attention heads to improve feature extraction.

Answer

It compresses the model weights to reduce storage requirements.

Answer

It replaces the attention mechanism with a recurrent neural network.

Question 4

When optimizing a Generative AI model using NVIDIA TensorRT-LLM, which component is primarily responsible for managing the KV cache to minimize memory fragmentation?

Accepted Answer

The Paged KV Cache Manager. TensorRT-LLM utilizes PagedAttention or similar paged memory management techniques to handle the Key-Value (KV) cache. By dividing the cache into fixed-size blocks, the framework avoids the fragmentation issues inherent in contiguous memory allocation. This ensures that the memory is used efficiently during autoregressive generation, allowing for larger context windows and higher concurrent user capacity on the GPU.

Answer

The TensorRT Builder

Answer

The Model Parallelism Engine

Answer

The Triton Inference Server

Question 5

A financial institution uses NVIDIA NeMo Guardrails to enforce ethical guidelines in its customer-facing LLM. During testing, the model occasionally generates responses that violate the company's policy against offering investment advice. The guardrails are configured with a set of dialog flows and safety checks. What is the most effective way to address this issue?

Accepted Answer

Enhance the NeMo Guardrails configuration with a custom action that checks the model's output against a compliance rule set and triggers a safe fallback response when a violation is detected.. The most effective solution is to enhance NeMo Guardrails with a custom action that evaluates the model's output against compliance rules and triggers a safe fallback. This leverages the extensibility of NeMo Guardrails to enforce policy dynamically and reliably, ensuring that any investment advice is intercepted and replaced.

Answer

Fine-tune the base LLM on a dataset of compliant responses to reduce the likelihood of generating investment advice.

Answer

Implement a post-processing filter that uses a separate LLM to classify responses as advice or non-advice and redacts them accordingly.

Answer

Add a custom guardrail that detects and blocks any mention of specific financial terms like 'invest' or 'stock'.

Question 6

Refer to the exhibit. Which prompt engineering technique would best force the model to prioritize technical detail over marketing language?

Accepted Answer

Add a constraint: 'Exclude marketing language and include specific architectural metrics like TDP, memory bandwidth, and interconnect speeds.'. The model is failing to adhere to the implicit expectation of 'technical depth.' By explicitly defining a structural constraint—such as requiring specific architectural metrics or removing marketing terminology—the model is forced to prioritize the technical aspects requested. This type of constraint-driven prompting is essential for professional NVIDIA technical writers and engineers who need precise documentation summaries.

Answer

Ask the model to 'Write a shorter summary' in the user prompt.

Answer

Decrease the temperature to 0.0 to make the model more factual.

Answer

Use few-shot prompting with generic summaries.

Question 7

When building an NVIDIA NeMo LLM application for automated document review, which THREE of the following prompt design choices are critical for ensuring high-quality output? (Select exactly THREE)

Accepted Answer

Use standardized delimiters to isolate user-provided documents.. Effective document review requires accuracy, traceability, and consistency. Using clear delimitation prevents data corruption, requiring structured output (like JSON) allows for downstream programmatic integration, and chain-of-thought prompts ensure the model validates its conclusions against the text. These choices together create a robust, production-ready pipeline that minimizes errors and provides the necessary structure for automated workflows in enterprise environments.

Answer

Include instructions that prioritize speed over accuracy.

Answer

Use a high temperature setting to ensure diverse review perspectives.

Question 8

Which THREE architectural features are essential for enabling efficient inference of massive LLMs on multi-GPU systems?

Accepted Answer

KV Caching to store previous sequence states.. Efficient inference at scale requires techniques that address memory constraints, compute latency, and communication overhead. Model parallelism, KV caching, and weight quantization are fundamental pillars that allow large models to fit within limited memory pools while maintaining high throughput. These techniques collectively ensure that the model remains responsive and cost-effective when serving large-scale requests in a production environment.

Answer

Batch Size reduction to increase memory throughput.

Answer

Increasing the learning rate during inference.

Question 9

When fine-tuning an LLM to follow specific safety protocols, why is the inclusion of 'adversarial' examples in the training data considered a best practice?

Accepted Answer

To ensure the model learns to refuse requests that violate established safety policies.. Adversarial examples test the model's ability to maintain safety boundaries even when prompted with manipulative or malicious inputs. By training on these examples, the model learns to identify and refuse requests that violate safety protocols. This proactive preparation is essential for deploying LLMs in enterprise environments where maintaining strict safety and compliance standards is required, protecting against sophisticated prompt injection and bypass techniques.

Answer

To artificially inflate the model's perplexity scores during evaluation.

Answer

To increase the model's fluency in generating complex technical instructions.

Answer

To allow the model to learn and reproduce the adversarial techniques during generation.

Question 10

Refer to the exhibit. An engineer observes that memory utilization spikes during inference, causing OOM errors. Given the configuration, what is the most likely cause of the failure?

Accepted Answer

The instance group count of 2 causes redundant loading of weights, exceeding VRAM.. The instance group configuration defines two concurrent instances on the same GPU. Each instance attempts to load a separate copy of the model weights into the GPU memory. If the model size is large, doubling the instances exceeds the available VRAM capacity. This configuration is a common mistake when deploying LLMs where model footprint is significant relative to total available device memory.

Answer

The count of 2 causes the GPU to oversubscribe its thermal limits.

Answer

Triton requires KIND_CPU for concurrent instance execution.

Answer

The gpus index [0] is invalid for multi-instance deployment.

Question 11

A production LLM service on NVIDIA Triton Inference Server uses dynamic batching. During peak load, the 99th percentile latency increases significantly, but GPU utilization remains at 60%. Which configuration change is most likely to improve latency while maintaining throughput?

Accepted Answer

Decrease the maximum batch size and adjust the batch timeout to reduce waiting. When GPU utilization is low but latency is high, the bottleneck is often the batching mechanism waiting to accumulate requests. Reducing the maximum batch size and batch timeout allows requests to be processed sooner, lowering queue time and P99 latency. Since the GPU has spare capacity, this change can improve latency without sacrificing throughput.

Answer

Enable model instances to increase parallelism on the same GPU

Answer

Switch to a larger GPU with more memory bandwidth

Answer

Increase the maximum batch size in the dynamic batching configuration

Question 12

A production LLM inference service on NVIDIA Triton Inference Server runs on a multi-GPU node. You observe that one GPU reports ECC XID errors and the model's throughput gradually degrades. Which NVIDIA tool should you use to monitor GPU health and set up alerts for these errors?

Accepted Answer

NVIDIA Data Center GPU Manager (DCGM). NVIDIA DCGM is the standard tool for monitoring GPU health in data centers. It tracks ECC and XID errors, temperature, power, and other metrics, and can trigger alerts. For a production LLM service experiencing hardware errors, DCGM provides the necessary observability to detect and respond to GPU failures.

Answer

NVIDIA TensorRT

Answer

NVIDIA Nsight Systems

Answer

NVIDIA Triton Model Analyzer

Question 13

A team is preparing a mixed-language corpus for continued pretraining of an NVIDIA NeMo Megatron model. The corpus contains English, Japanese, and Arabic documents. Tokenizer analysis shows the current English-centric BPE vocabulary produces very long token sequences for Japanese and Arabic, inflating sequence length and compute cost. The team wants to reduce sequence length for non-English text without retraining the tokenizer from scratch and without degrading English performance. Which data preparation action best achieves this?

Accepted Answer

Extend the existing BPE vocabulary with additional merges learned from a balanced multilingual sample, then retrain only the embedding and output layers while freezing the rest of the model.. The high token counts for Japanese and Arabic stem from a vocabulary that lacks subword units for those scripts. Extending the BPE vocabulary with multilingual merges shortens their token sequences while preserving English merges, and retraining only the embedding and output layers adapts the new entries without perturbing the pretrained transformer. This lowers compute cost and keeps English behavior stable.

Answer

Lowercase and strip diacritics from all Japanese and Arabic text so that the English-centric vocabulary can represent it more efficiently.

Answer

Transliterate all Japanese and Arabic documents into Latin script using a standard romanization scheme before tokenization.

Answer

Increase the model's maximum sequence length and positional embedding size so that long token sequences for Japanese and Arabic fit without truncation.

Question 14

Which THREE factors significantly influence the memory consumption during LLM fine-tuning? (Choose three)

Accepted Answer

The precision of the optimizer states. Memory consumption is driven by the model parameters, the optimizer states (which are often larger than the model weights), and the activations generated during the forward pass. Efficient management of these components is critical for scaling to larger models. By optimizing these three areas, practitioners can successfully fit large models into limited VRAM, ensuring stability throughout the fine-tuning process.

Answer

The number of CPU cores available for data loading

Answer

The version of the operating system kernel

Question 15

A bank's model risk committee is reviewing an LLM-based loan-adverse-action notice generator built on NVIDIA NeMo. Regulators require that the system produce a human-readable rationale for each denial and that the rationale be reproducible for any prior decision. Which architectural choice most directly meets both obligations?

Accepted Answer

Persist the exact prompt, model version, decoding parameters, retrieved context, and random seed for every decision so the notice can be regenerated identically.. Auditable generative decisions require capturing the full provenance chain: prompt, model version, decoding configuration, retrieval context, and seed. Storing those artifacts lets the bank reconstruct any prior notice byte-for-byte during an examination while keeping the natural-language rationale the regulation demands. Changing the model class, raising temperature, or merely caching outputs each fails one of the two obligations the committee must satisfy.

Answer

Increase the model temperature so the generator explores multiple rationales and select the most favorable one for each applicant.

Answer

Cache the generated notice text in a database keyed by applicant ID and serve the cached copy whenever the same applicant is re-scored.

Answer

Replace the LLM with a logistic regression scorecard so that every denial has a coefficient-based explanation by construction.

Question 16

Refer to the exhibit. The engineer is attempting to deploy on an NVIDIA Orin platform but encounters a runtime error. What is the most likely cause of the failure?

Accepted Answer

Unsupported operators are being targeted for the DLA. The inclusion of '--dla 0' forces the engine to run on the Deep Learning Accelerator (DLA) core. Many complex LLM operations, such as specific activation functions or advanced attention mechanisms, are not supported by the DLA's fixed-function logic. If the model graph contains unsupported operators, the build will either fail or generate a non-functional plan, as the DLA has a more restricted operator set than the primary GPU cores.

Answer

The workspace memory is too large

Answer

The DLA hardware is not enabled on this device

Answer

FP16 precision is not supported on DLA

Question 17

Refer to the exhibit. Which adjustment is the most immediate and effective way to resolve this OOM error while maintaining the same training architecture?

Accepted Answer

Decrease the batch size per GPU.. The error indicates that the current batch size is too large for the available VRAM. Reducing the batch size is the most direct solution to free up enough memory for the model to continue training. If a larger effective batch size is required for convergence, the user can subsequently enable gradient accumulation to compensate for the reduced per-step batch size without increasing the memory footprint.

Answer

Increase the sequence length of the inputs.

Answer

Switch from mixed precision to full FP32 precision.

Answer

Disable the optimizer state checkpointing.

Question 18

When optimizing a Transformer-based model using NVIDIA TensorRT, which TWO steps are essential to enable the Fusion of layer normalization and activation kernels to improve performance?

Accepted Answer

Enable the TensorRT optimization profile for layer fusion.. Kernel fusion combines multiple sequential operations into a single GPU kernel, reducing the overhead of launching many small kernels and minimizing global memory access. By fusing layer normalization and activation layers, the system avoids writing intermediate results back to main memory. This optimization is a core component of the TensorRT graph optimization phase, which analyzes the model architecture to identify and collapse redundant operations for better hardware utilization.

Answer

Ensure the model is defined using static input shapes.

Answer

Manually split the model into smaller sub-graphs.

Answer

Increase the precision to FP64 for better stability.

Question 19

Which technique provides the most significant performance gain for large-model inference by splitting the model across multiple GPUs while keeping the sequence length per batch constant?

Accepted Answer

Pipeline Parallelism.. Pipeline parallelism splits the model layers across multiple GPUs, allowing different layers of the model to be processed concurrently on different hardware. This is essential for models that are too large to fit into the memory of a single GPU. It enables the deployment of massive models by distributing the VRAM load, which is a foundational concept for scaling inference in production generative AI pipelines.

Answer

Data Parallelism.

Answer

Kernel Over-subscription.

Answer

Increasing global memory clock speed.

Question 20

Which THREE of the following are valid methods to optimize NVIDIA GPU memory bandwidth usage in deep learning?

Accepted Answer

Utilizing Operator Fusion.. Optimizing memory bandwidth is essential for memory-bound LLM tasks. Techniques like memory pooling, operator fusion, and precision reduction directly target the volume and frequency of data movement. These strategies ensure that data stays on the GPU as long as possible and that the data transferred is as small as possible, which is the key to maintaining high throughput in memory-constrained environments.

Answer

Aggressive use of FP64 precision.

Answer

Reducing the number of CUDA Streams.