NCP-GENL Practice Test 14 — 15 Questions

Question 1

An engineer is using NVIDIA TensorRT-LLM's in-flight batching to serve a mix of short and very long prompts. They observe that GPU utilization drops and latency for short requests spikes whenever a long prompt is admitted. Which mechanism should they tune to prevent long sequences from monopolizing the batch?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Adjust the maximum number of tokens processed per iteration and the KV cache block allocation policy to limit how much context a single request can consume.. In-flight batching improves throughput by mixing prefill and decode work, but a very long prompt can consume the entire per-iteration token budget, starving shorter sequences. Limiting tokens per iteration and controlling KV cache block allocation prevents any single request from monopolizing the batch. This restores GPU utilization and keeps short-request latency stable while still allowing long prompts to complete progressively.

Answer

Enable tensor parallelism so the long prompt's attention computation is split across multiple GPUs.

Answer

Switch from in-flight batching to a static batching scheme that groups requests by similar prompt length.

Answer

Increase the maximum batch size so more short requests can be admitted alongside the long prompt.

Question 2

A developer is building a customer support assistant using an NVIDIA NIM microservice for a Llama 3.1 8B Instruct model. The assistant must answer questions about an order solely based on a JSON payload containing order details, and it must not use any outside knowledge. Which prompt engineering approach best ensures the model adheres to this constraint?

Accepted Answer

Include the JSON payload in the prompt and instruct the model to answer using only the provided data, adding a fallback phrase for missing information.. Grounding the model with the exact JSON payload and instructing it to answer only from that data, with a fallback for missing information, ensures responses are based solely on the provided order details. This leverages the instruction-following ability of Llama 3.1 and avoids reliance on external knowledge. Other methods like fine-tuning or temperature adjustment do not enforce strict data adherence at inference time.

Answer

Prefix the prompt with a system message that says 'You are a helpful assistant' and then ask the question without including the JSON.

Answer

Fine-tune the model on a dataset of order-related questions and answers before deploying it as a NIM microservice.

Answer

Use a low temperature setting (e.g., 0.1) to make the model more deterministic and less likely to hallucinate.

Question 3

A developer is building a customer support assistant using an NVIDIA NIM microservice for a Llama 3 model. The assistant must always respond in valid JSON with keys 'category' and 'urgency'. The model often returns conversational text instead. Which prompt engineering change most directly enforces the required output format?

Accepted Answer

Append 'Return only JSON with keys category and urgency' and set the NIM request parameter 'guided_json' to the target schema.. Structured output requires both a clear instruction and a decoding-time constraint. NVIDIA NIM supports guided_json, which restricts token generation to a supplied JSON schema, guaranteeing valid keys and syntax. Pairing that with an explicit prompt instruction aligns the model's behavior with the schema. Few-shot examples or temperature changes alone cannot guarantee strict JSON compliance.

Answer

Add 'Do not hallucinate' to the prompt and lower the max_tokens parameter to 50.

Answer

Add a system prompt that says 'You are a helpful assistant' and increase the temperature to 0.9.

Answer

Use few-shot examples of JSON outputs and set top_p to 0.1 without any schema constraint.

Question 4

An engineer is deploying a 13B-parameter LLM with TensorRT-LLM on a single NVIDIA A100 40GB GPU. The FP16 engine requires 26GB for weights, but during generation the KV cache grows beyond remaining memory, causing out-of-memory errors. The team wants to maximize concurrent requests without retraining. Which optimization should they apply first?

Accepted Answer

Reduce the KV cache memory footprint by quantizing the KV cache to INT8 while keeping model weights in FP16.. The KV cache grows linearly with sequence length and batch size, and it is the dominant memory consumer during LLM generation after weights are loaded. Quantizing only the KV cache to INT8 halves its footprint without retraining and without changing weight precision, directly relieving the OOM. This keeps the model on the existing A100 while allowing more concurrent requests, which matches the stated goal.

Answer

Increase the batch size and sequence length limits so the scheduler can pack more requests into the same memory.

Answer

Enable tensor parallelism across two A100 GPUs to split both weights and KV cache across devices.

Answer

Convert the model weights from FP16 to FP8 using post-training quantization to free memory for the KV cache.

Question 5

An engineer is training a large language model with pipeline parallelism across four NVIDIA GPUs. They observe that GPU utilization is low and training throughput is limited by idle time during pipeline bubbles. Which technique is most effective to reduce pipeline bubbles and improve utilization?

Accepted Answer

Increase the number of micro-batches to keep the pipeline stages busy.. Pipeline bubbles are idle periods when stages wait for data. Increasing the number of micro-batches allows more fine-grained overlap of computation across stages, keeping all GPUs busy and reducing idle time. This is a direct and effective way to improve pipeline utilization.

Answer

Reduce the batch size to decrease the memory footprint and allow more concurrent kernels.

Answer

Enable gradient checkpointing to trade compute for memory and increase batch size.

Answer

Switch from pipeline parallelism to data parallelism across the four GPUs.

Question 6

A financial services company has fine-tuned a Llama-3-70B model using NVIDIA NeMo for automated loan risk assessment. During evaluation, they observe that the model achieves high scores on standard accuracy metrics but produces inconsistent responses when the same query is phrased slightly differently. They need a metric that quantifies this inconsistency. Which evaluation metric should they use?

Accepted Answer

Semantic similarity between outputs for paraphrased inputs. The team needs to measure output consistency across semantically equivalent inputs. Semantic similarity between responses to paraphrased queries directly captures this variance, unlike lexical overlap metrics such as BLEU or ROUGE, which compare to references. Perplexity reflects language modeling confidence, not answer stability. Therefore, semantic similarity is the correct choice for quantifying inconsistency in the fine-tuned model's responses.

Answer

ROUGE score

Answer

BLEU score

Answer

Perplexity

Question 7

A developer is using an NVIDIA NIM for a Llama 3.1 70B model to build a legal document review assistant. The model must answer questions based on a provided contract, but the contracts are often 50,000 tokens long, exceeding the model's 8,000-token context window. Which prompt engineering strategy is most appropriate to handle this constraint?

Accepted Answer

Use a retrieval-augmented generation (RAG) approach to fetch only the relevant sections of the contract and include them in the prompt.. RAG is the most appropriate strategy because it retrieves only the relevant sections of the long contract, fitting them into the model's context window. This allows the model to answer questions accurately without needing to process the entire document. It is a widely adopted prompt engineering pattern for handling context length limitations.

Answer

Split the contract into chunks and ask the model to summarize each chunk sequentially, then combine the summaries.

Answer

Increase the model's context window by fine-tuning it on longer sequences.

Answer

Use a sliding window approach where the model processes the contract in overlapping segments and maintains a memory of previous segments.

Question 8

An engineer is optimizing a large language model for inference on NVIDIA GPUs and wants to reduce memory usage to fit a larger model or increase batch size. Which two techniques are most effective for reducing GPU memory consumption during inference? (Choose two.)

Accepted Answer

Quantize model weights to 8-bit integers using TensorRT or similar tools.. Quantizing weights to INT8 reduces memory footprint by storing weights in lower precision, while paged attention optimizes KV cache memory by reducing fragmentation. Both are effective for fitting larger models or increasing batch size during inference. Activation checkpointing and micro-batching are training or throughput techniques, and CPU offloading introduces latency.

Answer

Enable activation checkpointing to recompute activations during the backward pass.

Answer

Increase the number of micro-batches to overlap computation and memory transfers.

Answer

Store the model weights on the CPU and transfer them to the GPU on demand.

Question 9

A team is training a large language model on a single NVIDIA H100 GPU. They observe that training throughput is significantly lower than expected, and profiling with Nsight Systems shows long periods where the GPU is idle waiting for data. The data loading pipeline uses the default PyTorch DataLoader with num_workers=0 and no pinned memory. Which change is most likely to improve GPU utilization?

Accepted Answer

Enable pinned memory and increase num_workers in the DataLoader.. The profiling evidence points to a data-loading bottleneck: the GPU sits idle waiting for batches. Enabling pinned memory and increasing DataLoader workers allows asynchronous, overlapped data transfer and prefetching, keeping the GPU fed. The other options target compute or launch overhead, which are not the limiting factors in this scenario.

Answer

Use CUDA graphs to capture the training step and reduce kernel launch overhead.

Answer

Switch from FP32 to TF32 precision for matrix multiplications.

Answer

Increase the batch size to better saturate the GPU.

Question 10

An engineer is optimizing a large language model for inference on NVIDIA GPUs using TensorRT-LLM. They want to reduce the memory footprint of the KV cache to support longer context lengths and more concurrent requests. Which two techniques should they implement? (Choose two.)

Accepted Answer

Enable paged KV cache with block-based memory allocation.. INT8 quantization and paged KV cache are both designed to reduce memory footprint. Quantization lowers the bit-width of stored keys/values, while paging eliminates fragmentation and allows more efficient allocation. Together, they enable longer contexts and higher concurrency without increasing GPU memory.

Answer

Use FP32 precision for the KV cache to avoid accuracy loss.

Answer

Enable multi-head attention with larger head dimension.

Answer

Increase the batch size to improve memory reuse.

Question 11

A team is building a TensorRT-LLM engine for a 7B model that must serve both single-turn short prompts and long multi-turn conversations with a shared system prompt. They want to maximize reuse of computation across requests without changing model weights. Which TWO techniques should they enable? (Choose two.)

Accepted Answer

Enable paged KV cache with block reuse so freed sequence blocks return to the pool for other requests.. Sharing a system prompt and multi-turn history means many requests share long identical prefixes, and mixed short and long prompts create variable cache pressure. KV cache reuse skips recomputing cached prefixes, while the paged KV cache with block reuse keeps memory defragmented and available to new sequences. Together they cut redundant prefill and raise concurrency without touching model weights.

Answer

Apply INT4 weight-only quantization to the attention projection matrices to shrink the cache footprint.

Answer

Set the builder to use strongly typed engines so the plugin graph is fully deterministic.

Answer

Increase the beam width so multiple candidate continuations are evaluated per request.

Question 12

A team is using an NVIDIA NeMo-based LLM to answer questions over a product manual. The model sometimes answers using general knowledge instead of the provided manual excerpts. They want to force the model to rely only on the supplied context. Which prompt engineering approach best addresses this?

Accepted Answer

Instruct the model to answer only from the provided context and to reply 'Not in the provided context' when the answer is absent.. Grounding a model in retrieved context requires an explicit instruction that restricts answers to that context and defines what to do when the answer is missing. A fallback phrase such as 'Not in the provided context' prevents the model from filling gaps with pretrained knowledge. Few-shot examples help style but do not enforce the boundary; temperature and context removal work against the goal.

Answer

Increase the model's temperature so it explores more diverse answers from its pretrained knowledge.

Answer

Add more few-shot examples of correct answers without changing the instruction about using the context.

Answer

Shorten the prompt by removing the manual excerpts and rely on the model's product knowledge.

Question 13

A developer wants to reduce the disk and memory footprint of a fine-tuned 70B model before serving it with TensorRT-LLM, and is willing to accept a small, measurable quality drop that they will validate with an evaluation harness. Which approach best matches that requirement?

Accepted Answer

Perform post-training quantization to INT4 or INT8 weights and validate with perplexity and task metrics.. The requirement is a smaller weight footprint with an accepted, validated quality trade-off and no retraining. Post-training quantization of the fine-tuned checkpoint to INT4 or INT8 delivers that reduction immediately and pairs naturally with an evaluation harness to confirm the quality delta stays within tolerance. Retraining-based methods exceed the stated scope.

Answer

Retrain the model from scratch using quantization-aware training with fake-quant nodes in the forward pass.

Answer

Convert the checkpoint to FP8 and rely on the runtime to upcast to FP16 for every matmul.

Answer

Increase the tensor parallelism degree so each GPU holds a smaller shard of the FP16 weights.

Question 14

A data scientist is using an NVIDIA NeMo LLM to generate Python code from natural language descriptions. The model often produces code that works but does not follow the team's style guide, such as using single quotes instead of double quotes and missing type hints. Which prompt engineering technique should the data scientist use to improve adherence to the style guide?

Accepted Answer

Few-shot prompting with examples that demonstrate the desired coding style.. Few-shot prompting with examples that embody the desired style guide is the most effective way to teach the model the specific formatting rules. By showing the model correct examples, it can mimic the style in new generations. This technique is particularly useful for coding tasks where precise syntax and style matter.

Answer

Chain-of-thought prompting to encourage the model to reason about the style guide before writing code.

Answer

Zero-shot prompting with a detailed instruction describing the style guide rules.

Answer

Increasing the temperature to allow more creative code generation.

Question 15

An engineer is deploying an NVIDIA NeMo Guardrails system to moderate a chatbot's responses. The chatbot must refuse to answer questions about politics but should answer questions about weather. Which prompt engineering strategy in NeMo Guardrails is most appropriate to enforce this behavior?

Accepted Answer

Define a dialogue flow with a canonical form for political questions that triggers a refusal response, and a separate flow for weather questions that allows the answer.. NeMo Guardrails uses Colang dialogue flows to define bot behavior based on user intent. Creating separate flows for political and weather questions ensures deterministic refusal or answering. This is the core prompt engineering approach within the guardrails framework and provides reliable, auditable control.

Answer

Add a Python function that checks the user input for political keywords and returns a refusal, bypassing the model entirely.

Answer

Use a system prompt that instructs the model to refuse political questions and answer weather questions.

Answer

Fine-tune the underlying LLM to refuse political questions and answer weather questions.