NCP-GENL Practice Test 13 — 15 Questions

Question 1

A team is deploying a large language model using NVIDIA TensorRT-LLM on a multi-GPU node with NVLink. They want to minimize inter-GPU communication overhead during inference. Which parallelism strategy should they use to achieve this?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Tensor parallelism (TP). Tensor parallelism splits layers across GPUs and uses all-reduce for activations. On a system with NVLink, the high bandwidth and low latency of NVLink make TP efficient, minimizing communication overhead. Pipeline parallelism introduces bubbles, data parallelism replicates the model, and expert parallelism is for MoE models and incurs all-to-all communication.

Answer

Pipeline parallelism (PP)

Answer

Expert parallelism (EP) in a Mixture-of-Experts model

Answer

Data parallelism (DP)

Question 2

You are curating a 2 TB corpus of NVIDIA technical documentation and Python code for continued pretraining of a NeMo-based LLM. A colleague proposes filtering out any document containing the token sequence 'CUDA' to reduce hardware-specific bias. What is the most appropriate response?

Accepted Answer

Reject the filter, because removing documents based on a single high-signal domain token destroys the domain-specific signal the model needs and is better handled by deduplication and quality scoring.. Keyword-based deletion of a domain-defining token is a destructive filter, not a quality filter. Effective NeMo data curation relies on deduplication, quality heuristics, and classifier scoring to remove low-value or redundant records while preserving coherent, in-domain technical content. The corpus exists to teach NVIDIA-specific concepts, so removing 'CUDA' would directly undermine the training objective.

Answer

Accept the filter, because removing every occurrence of a hardware-specific token guarantees the model will not overfit to NVIDIA-specific APIs and will generalize better to other vendors.

Answer

Accept the filter but apply it only to documents where 'CUDA' appears more than ten times, since low-frequency occurrences are harmless and high-frequency ones indicate redundant marketing material.

Answer

Accept the filter, because NVIDIA NeMo requires that proprietary product names be excluded from pretraining corpora to comply with the model card's data provenance requirements.

Question 3

An engineer is using NVIDIA TensorRT-LLM to optimize an LLM for inference. They want to reduce the memory footprint of the KV cache during long-context generation. Which TWO techniques are supported by TensorRT-LLM to achieve this? (Choose two.)

Accepted Answer

Enable paged KV cache with block sharing.. TensorRT-LLM provides native support for FP8 KV cache quantization and paged KV cache with block sharing. FP8 quantization halves cache memory, while paged cache with block sharing reduces duplication across sequences. Both are configuration-time features that do not require model changes. The other options involve model modifications or misapply TensorRT features, making them incorrect for this scenario.

Answer

Use TensorRT's INT8 calibration for the KV cache.

Answer

Apply structured pruning to remove attention heads.

Answer

Enable sliding window attention in the model architecture.

Question 4

A team is fine-tuning a Llama 2 7B model with NVIDIA NeMo Framework on a single A100 80GB GPU. They observe that training loss decreases initially but then diverges, and the model outputs become repetitive and incoherent. The team used a learning rate of 5e-5 with AdamW and no warm-up. Which change is most likely to stabilize training and improve convergence?

Accepted Answer

Reduce the learning rate to 1e-5 and add a linear warm-up over the first 10% of training steps.. The model diverges due to a learning rate that is too high for fine-tuning without warm-up. Reducing the learning rate to 1e-5 and adding a warm-up phase allows the optimizer to take smaller, more controlled steps initially, preventing large updates that disrupt pretrained weights. This combination is a well-established best practice in NVIDIA NeMo and other frameworks for stable fine-tuning of LLMs.

Answer

Increase the learning rate to 1e-4 to escape local minima.

Answer

Switch the optimizer to SGD with momentum 0.9 and keep the learning rate at 5e-5.

Answer

Increase the batch size to 64 and keep all other hyperparameters unchanged.

Question 5

An engineer is building a customer support assistant using an NVIDIA NIM for a Llama 3 70B model. The assistant must always respond in valid JSON containing exactly the keys "issue" and "urgency", and must never include any other text. Which prompt engineering approach most directly enforces this output contract?

Accepted Answer

Add a system message that specifies the JSON schema and instructs the model to output only JSON, then validate responses programmatically.. The most direct way to enforce a strict JSON contract is to state the schema and the output-only requirement in the system message, which has the highest priority in the prompt hierarchy, and then validate the result programmatically. Sampling parameters and stylistic instructions do not constrain structure, so they cannot guarantee the two required keys or the absence of extra text.

Answer

Raise the temperature to 1.0 so the model explores more possible response formats and eventually produces valid JSON.

Answer

Append the phrase "Please be concise" to every user message and rely on the model to infer the JSON structure.

Answer

Set the top_p value to 0.1 and the max_tokens parameter to 50 to force the model into a JSON-only response mode.

Question 6

A developer is using NVIDIA TensorRT-LLM to optimize a GPT-based model for inference. They want to reduce the model's memory footprint and improve throughput without retraining. Which two techniques can be applied during the TensorRT-LLM build process to achieve these goals? (Choose two.)

Accepted Answer

Enabling FP8 precision for both weights and activations on Hopper GPUs. Weight-only quantization and FP8 precision are both build-time optimizations in TensorRT-LLM that reduce memory footprint and improve throughput without retraining. Weight-only quantization lowers weight precision, while FP8 leverages Hopper GPU capabilities for both weights and activations. The other options either require retraining or increase memory usage.

Answer

Pruning the model by removing entire attention heads

Answer

Knowledge distillation from a larger teacher model

Answer

Using a larger batch size to amortize memory overhead

Question 7

An engineer is deploying a large language model using NVIDIA TensorRT-LLM on an A100 GPU. The model uses multi-head attention with a sequence length of 4096. During inference, the GPU's Tensor Cores are underutilized, and the kernel launch overhead is high due to many small operations. Which optimization should be applied to improve Tensor Core utilization and reduce overhead?

Accepted Answer

Enable kernel fusion by using TensorRT-LLM's fused multi-head attention plugin.. The underutilization of Tensor Cores and high kernel launch overhead stem from many small operations in multi-head attention. TensorRT-LLM's fused multi-head attention plugin combines these operations into a single optimized kernel, reducing overhead and improving Tensor Core utilization. This is a targeted optimization for Transformer models, especially with long sequences.

Answer

Increase the batch size to maximize parallelism and hide latency.

Answer

Use NVIDIA Triton Inference Server with dynamic batching to improve GPU utilization.

Answer

Convert the model to FP16 precision to double the Tensor Core throughput.

Question 8

A team is fine-tuning a 13B-parameter Llama model with NVIDIA NeMo on a node of eight A100 80GB GPUs. They want the optimizer state to be partitioned across data-parallel ranks so that per-GPU memory drops, while keeping the model replicas synchronized. Which distributed strategy should they select in the NeMo training configuration?

Accepted Answer

ZeRO Stage 1 via the distributed optimizer in Megatron Core. Sharding optimizer state across data-parallel ranks is exactly what the distributed optimizer in Megatron Core does, providing ZeRO Stage 1 behavior with full model replicas per rank. Tensor and pipeline parallelism change how parameters and layers are split, and recomputation targets activations. Only optimizer-state partitioning reduces the Adam moments and master weights each GPU must hold without adding model-parallel communication.

Answer

Tensor Parallelism (TP) with tensor_model_parallel_size set to 8

Answer

Pipeline Parallelism (PP) with pipeline_model_parallel_size set to 8

Answer

Activation recomputation with recompute_granularity set to full

Question 9

A developer is optimizing a BERT-based model for inference on an NVIDIA T4 GPU using TensorRT. The model has a fixed input sequence length of 128. Profiling shows that the kernel execution time is high due to many small operations. Which TensorRT feature should they use to reduce kernel launch overhead and improve latency?

Accepted Answer

Apply CUDA graphs to capture the entire inference graph and replay it with a single launch.. CUDA graphs are designed to reduce kernel launch overhead by capturing a static sequence of operations and replaying it as a single unit. For a fixed-shape model like the BERT model with sequence length 128, the graph can be captured once and reused across inferences. This eliminates the CPU-side launch latency for each kernel, directly improving latency.

Answer

Increase the batch size to amortize kernel launch overhead across more samples.

Answer

Use TensorRT's builder optimization level 5 to enable aggressive layer fusion.

Answer

Enable FP16 precision and calibrate with a representative dataset.

Question 10

An engineer is deploying a large language model using NVIDIA TensorRT-LLM on an A100 GPU. They want to maximize throughput for a chatbot workload with variable-length inputs and outputs. Which of the following techniques should they implement to achieve the highest throughput while maintaining acceptable latency?

Accepted Answer

Enable in-flight batching (continuous batching) and use paged KV cache with block-based memory management.. In-flight batching and paged KV cache are key optimizations in TensorRT-LLM for LLM serving. In-flight batching allows sequences to join and leave the batch at each decoding step, avoiding padding and keeping the GPU busy. Paged KV cache manages memory in blocks, enabling more sequences to run concurrently. This combination delivers the highest throughput for variable-length chatbot workloads.

Answer

Use static batching with a fixed batch size of 32 and pad all sequences to the maximum length.

Answer

Use FP32 precision to ensure accuracy and rely on CUDA graphs to reduce launch overhead.

Answer

Increase the number of GPUs and use tensor parallelism with a fixed batch size per GPU.

Question 11

An engineer is optimizing a Transformer-based LLM for inference on an NVIDIA A100 GPU. The model uses FP16 precision, but during generation, the GPU's Tensor Cores are underutilized, and latency is higher than expected. Profiling reveals that many small matrix multiplications are executed sequentially. Which technique is most effective to improve Tensor Core utilization and reduce latency?

Accepted Answer

Apply kernel fusion to combine multiple small operations into a single kernel.. The underutilization of Tensor Cores stems from many small, sequential matrix multiplications. Kernel fusion combines these into larger operations, allowing Tensor Cores to process more data per instruction and reducing kernel launch overhead. This directly improves utilization and lowers latency, making it the most effective technique among the options for this specific bottleneck.

Answer

Enable CUDA graphs to capture and replay the sequence of kernel launches.

Answer

Use NVIDIA TensorRT to quantize the model to INT8.

Answer

Increase the batch size to amortize kernel launch overhead.

Question 12

A developer is building an interactive assistant using NVIDIA NIM microservices. The assistant must answer questions about a specific set of internal policies. The developer wants to ensure the model's responses are grounded in those policies and not in its general pre-training knowledge. Which prompt engineering technique should be applied?

Accepted Answer

Provide the relevant policy excerpts directly within the prompt, instruct the model to answer only from that context, and to state when the answer is not present.. Grounding the model in a specific set of documents requires placing the relevant excerpts in the prompt and explicitly instructing the model to answer only from that context. This constrains the model to the provided facts and reduces hallucination. Other techniques like increasing temperature, chain-of-thought, or fine-tuning do not achieve this grounding for a dynamic, interactive policy assistant.

Answer

Fine-tune the model on the internal policy documents, then use a simple zero-shot prompt to ask questions.

Answer

Use a chain-of-thought prompt that asks the model to reason step by step about the policy before answering.

Answer

Increase the temperature setting to 0.9 to encourage the model to explore a wider range of policy interpretations.

Question 13

A developer is using NVIDIA TensorRT to optimize a BERT-based model for inference. They notice that the engine performs poorly on variable-length input sequences because it was built with a single optimization profile for a fixed sequence length. What should they do to improve performance across different sequence lengths?

Accepted Answer

Rebuild the engine with multiple optimization profiles covering the range of sequence lengths.. TensorRT optimization profiles allow the engine to tune kernels for specific input shape ranges. When input lengths vary, a single profile cannot cover all cases efficiently. Creating multiple profiles, each with appropriate min/opt/max dimensions, enables the engine to choose the best kernels for each length, improving overall performance.

Answer

Pad all input sequences to the maximum length supported by the model.

Answer

Use CUDA graphs to capture and replay the inference execution.

Answer

Enable FP16 precision to reduce memory usage and speed up kernels.

Question 14

A team is building an instruction-tuning dataset in NeMo from 40,000 internal support tickets. Each ticket contains a customer problem and a resolved answer, but the resolution text sometimes includes the customer's name, account number, and internal case IDs. The team plans to use NeMo Curator to produce training-ready JSONL. Which approach best prepares this data for instruction tuning while limiting personally identifiable information exposure?

Accepted Answer

Run a PII redaction stage that detects and replaces names, account numbers, and case IDs with placeholders, then convert problem-resolution pairs into the instruction, input, and output schema before export.. Sensitive identifiers must be removed from the corpus before training, not mitigated after the fact. Running a PII detection and replacement stage in NeMo Curator, then mapping problem-resolution pairs into the instruction, input, and output schema, produces compliant JSONL that the fine-tuning job can consume directly. This ordering keeps identifiers out of both the training data and the resulting model weights.

Answer

Hash the entire ticket text with a cryptographic digest and train on the hashes, since the model can learn the mapping between hashed problems and hashed answers without ever seeing the original identifiers.

Answer

Convert the tickets into instruction, input, and output JSONL first, then fine-tune the model and rely on a post-training output filter to block any generated response that contains an account number or customer name.

Answer

Drop every ticket whose resolution contains any digit, since account numbers and case IDs always contain numeric characters, and train only on the remaining text-only resolutions.

Question 15

A developer is using NVIDIA Nsight Systems to profile a PyTorch training loop on an NVIDIA GPU. They notice significant gaps between kernel executions and want to identify whether the bottleneck is CPU-side or GPU-side. Which Nsight Systems feature should they use to visualize the CPU and GPU timelines together?

Accepted Answer

The CUDA API trace and GPU activity timeline.. Nsight Systems' unified timeline displays CPU activities (including CUDA API calls) and GPU kernels on the same time axis. This allows developers to see gaps where the GPU is idle while the CPU is busy, indicating a CPU-bound workload. It is the standard tool for this type of system-level analysis.

Answer

The Nsight Compute kernel profiling report.

Answer

The NVIDIA Management Library (NVML) GPU utilization metrics.

Answer

The PyTorch autograd profiler output.