NCP-GENL Practice Test 12 — 15 Questions

Question 1

An enterprise is running a mission-critical generative AI application on an NVIDIA DGX cluster. The MLOps team notices occasional silent GPU memory corruption during long-running inference jobs that do not trigger hard crashes. Which monitoring tool and strategy should be utilized for early detection?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Deploy NVIDIA Data Center GPU Manager (DCGM) with Prometheus exporter to monitor XID errors and hardware ECC events continuously.. DCGM (Data Center GPU Manager) provides specialized diagnostic tests and continuous health monitoring, including ECC error tracking and XID error detection. Configuring DCGM to raise alerts on uncorrectable memory errors or specific XID failure codes allows operators to isolate failing GPUs before they impact production workloads.

Answer

Increase the frequency of client-side HTTP ping probes sent from the load balancer to the API gateway.

Answer

Write a custom bash script that runs nvidia-smi every five minutes and parses plain text output for warning strings.

Answer

Rely on standard Kubernetes node liveness probes to automatically restart pods when system memory usage exceeds 90%.

Question 2

An engineer is deploying a large language model for real-time inference on an NVIDIA A100 GPU. The model uses FP16 weights but the inference server must handle variable-length input sequences. Profiling shows that the GPU spends significant time on memory-bound operations and that kernel launches are frequent. Which optimization is most appropriate to reduce latency while maintaining accuracy?

Accepted Answer

Use NVIDIA TensorRT with dynamic batching and CUDA graphs to capture the inference step.. TensorRT with dynamic batching and CUDA graphs targets the two identified issues: variable-length sequences are handled by dynamic shapes and batching, while CUDA graphs eliminate launch overhead by capturing the entire inference step. This approach maintains FP16 accuracy and improves latency without the risks of quantization.

Answer

Increase the number of CUDA streams to parallelize kernel execution.

Answer

Enable NVIDIA GPUDirect Storage to accelerate data loading.

Answer

Convert the model to INT8 using TensorRT with calibration.

Question 3

A technical support team is building a chatbot using an NVIDIA NIM microservice. The chatbot must answer questions about a specific product's warranty policy. The team wants to ensure the model's responses are grounded in the official warranty document, which is 50 pages long, and avoid inventing policy details. Which prompt engineering approach is most effective for this scenario?

Accepted Answer

Use a retrieval-augmented generation (RAG) pipeline to fetch the most relevant warranty sections and include them in the prompt as context.. Retrieval-augmented generation (RAG) is the most effective way to ground responses in a specific document. It retrieves relevant sections and includes them in the prompt, ensuring the model's answer is based on factual content rather than pre-trained memory. This reduces hallucinations and handles documents that exceed the context window. Other methods either risk hallucination, are inefficient, or lack flexibility.

Answer

Ask the model to 'think step by step' and reason about the warranty policy from its pre-trained knowledge.

Answer

Fine-tune the model on the warranty document using NVIDIA NeMo, then use the fine-tuned model without any additional context in the prompt.

Answer

Include the entire warranty document in the system prompt and instruct the model to answer based only on that document.

Question 4

A team has deployed a Llama-3-8B model with NVIDIA Triton Inference Server for real-time chatbot responses. They need to evaluate the model's generation quality on a held-out test set of 500 prompts, focusing on semantic similarity to reference answers while ensuring low latency. Which evaluation approach is most appropriate to use with NVIDIA NeMo Evaluator?

Accepted Answer

Use BERTScore or a similar embedding-based metric via NeMo Evaluator's custom metric interface.. For evaluating semantic similarity in generated text, embedding-based metrics like BERTScore are preferred because they capture meaning beyond exact wording. NeMo Evaluator's extensibility allows incorporating such metrics, providing a more human-aligned assessment for chatbot responses. Perplexity and n-gram overlap metrics like BLEU and ROUGE-L do not adequately measure semantic equivalence, especially when paraphrasing is expected.

Answer

Calculate perplexity on the test set using the model's own logits.

Answer

Compute BLEU score using the sacreBLEU library integrated into NeMo Evaluator.

Answer

Run ROUGE-L scoring on the generated outputs and references.

Question 5

A developer is prompting an NVIDIA NIM for a Code Llama model to generate a Python function. The model produces correct logic but frequently omits type hints and docstrings, which the team requires. Which prompting technique best addresses this specific gap?

Accepted Answer

Include a short example of the desired function signature, type hints, and docstring in the prompt, then ask for the new function in the same style.. A concrete example showing the desired function signature, type hints, and docstring teaches the model the exact format by imitation. Vague quality instructions, larger token budgets, and reasoning steps do not specify these structural requirements, so they fail to close the gap. The example-based approach is the most direct and reliable fix.

Answer

Increase max_tokens so the model has more room to include type hints and docstrings.

Answer

Add the instruction "write clean, production-quality code" to the system message.

Answer

Ask the model to first explain its reasoning about the function, then output the code.

Question 6

An engineer is optimizing a BERT-like model for inference using NVIDIA TensorRT. They want to reduce latency further by using lower precision without significant accuracy loss. Which TensorRT precision mode should they choose to enable INT8 inference while maintaining accuracy through calibration?

Accepted Answer

INT8. INT8 precision mode in TensorRT allows inference using 8-bit integers, reducing latency and memory footprint. To preserve accuracy, a calibrator computes scaling factors from a calibration dataset. The other precisions (FP32, FP16, TF32) do not provide INT8 inference and do not use calibration for quantization.

Answer

FP16

Answer

FP32

Answer

TF32

Question 7

A financial analyst is using an NVIDIA NIM-hosted Llama 3.1 70B model to extract key financial metrics from quarterly earnings call transcripts. The model inconsistently returns a prose summary instead of the required structured JSON. The analyst needs the output to be reliably parseable by a downstream script that expects a fixed schema with fields "revenue", "eps", and "guidance". Which prompt engineering technique is most appropriate to enforce this output format?

Accepted Answer

Provide a JSON schema and a completed example in the prompt, and instruct the model to return only JSON matching that schema.. Enforcing a structured output like JSON requires explicit specification of the schema and often a concrete example. Providing a schema and a completed example leverages the model's in-context learning to mimic the exact format, while the instruction to return only JSON reduces extraneous text. Other techniques like temperature adjustment or chain-of-thought do not directly control output structure and may even worsen format compliance.

Answer

Increase the temperature parameter to 0.9 so the model explores more diverse output formats and may eventually produce JSON.

Answer

Use a chain-of-thought prompt asking the model to 'think step by step' before answering, which will naturally lead to JSON output.

Answer

Add a system prompt that says 'You are a helpful assistant' and rely on the model's instruction-following ability to infer the JSON requirement.

Question 8

A team is serving a 70B-parameter LLM with TensorRT-LLM in a multi-tenant environment where requests arrive with widely varying prompt lengths and generation lengths. During load testing, they observe that throughput collapses when a long-context request is scheduled alongside many short requests, and GPU memory fragmentation causes intermittent out-of-memory errors even though total free memory appears sufficient. Which TensorRT-LLM runtime configuration change most directly addresses both the throughput collapse and the memory fragmentation?

Accepted Answer

Enable the paged KV cache with a tuned block size and configure the batch scheduler to use in-flight batching with a per-iteration token budget.. The paged KV cache solves external fragmentation by allocating KV memory in uniform blocks from a shared pool, which prevents the OOM that occurs when a long-context request needs a large contiguous region. In-flight batching with a token budget lets the scheduler continuously admit and retire requests each iteration, so short requests no longer wait behind a long generation. These two runtime settings together address both symptoms without sacrificing accuracy or throughput.

Answer

Increase the TensorRT-LLM build max_batch_size and max_seq_len to their maximum possible values so the engine supports every request shape.

Answer

Switch the engine from FP16 to INT8 precision using a calibration dataset that matches the production prompt distribution.

Answer

Reduce the number of concurrent client connections at the load balancer so that only one request is processed by the GPU at a time.

Question 9

An engineer is using an NVIDIA NIM for a code generation model to produce Python functions from natural language descriptions. The model frequently generates code that uses deprecated libraries or incorrect function signatures. The engineer wants to improve the accuracy of the generated code by providing examples. Which prompting strategy is most appropriate?

Accepted Answer

Few-shot prompting with examples that demonstrate the correct use of the desired libraries and function signatures.. Few-shot prompting with examples that demonstrate correct library usage and function signatures is the most direct way to guide the model to produce accurate code. By showing the desired pattern, the model can imitate it, reducing the likelihood of deprecated libraries or incorrect signatures. Other strategies like zero-shot, chain-of-thought, or self-consistency do not provide the concrete examples needed to correct systematic API errors.

Answer

Zero-shot prompting with a detailed description of the desired function and its parameters.

Answer

Self-consistency prompting where the model generates multiple code solutions and the most common one is selected.

Answer

Chain-of-thought prompting that asks the model to first explain its reasoning about which libraries to use, then write the code.

Question 10

A developer is optimizing a generative AI model for inference on NVIDIA GPUs. They want to reduce memory footprint and improve throughput without sacrificing accuracy. Which two techniques should they apply? (Choose two.)

Accepted Answer

Use NVIDIA TensorRT with INT8 quantization and calibration.. INT8 quantization and structured sparsity both reduce memory footprint and improve throughput. INT8 quantization uses 8-bit integers, cutting memory usage by 4x compared to FP32, and Tensor Cores accelerate INT8 operations. Structured sparsity prunes weights in a 2:4 pattern, halving memory for weights and enabling sparse Tensor Core acceleration. Both are supported by TensorRT and maintain accuracy with proper calibration and fine-tuning.

Answer

Use NVIDIA GPUDirect Storage to offload model weights to NVMe.

Answer

Increase the batch size to amortize memory overhead.

Answer

Enable NVIDIA Ampere TF32 precision for matrix multiplications.

Question 11

A team is serving a 13B-parameter LLM on a single NVIDIA A100 80GB GPU. During generation, they observe that the GPU compute utilization stays below 20% while memory bandwidth utilization is near saturation. They want to improve throughput without changing the model architecture. Which optimization is most appropriate?

Accepted Answer

Apply 4-bit weight-only quantization to reduce memory traffic during token generation.. The symptoms indicate a memory-bandwidth-bound workload, common in autoregressive LLM decoding. Reducing weight precision via 4-bit quantization decreases the volume of data read from GPU memory per token, directly increasing throughput. Other options either do not target memory bandwidth or introduce new bottlenecks.

Answer

Enable multi-GPU tensor parallelism across two A100 GPUs to split the model.

Answer

Increase the batch size to improve arithmetic intensity and better utilize tensor cores.

Answer

Increase the number of CPU threads used for token sampling to speed up generation.

Question 12

A generative AI application built on NVIDIA Triton Inference Server is deployed in a Kubernetes cluster with GPU nodes. The operations team wants to detect silent data corruption in model outputs, which could occur due to GPU memory errors. They plan to implement a monitoring solution using NVIDIA Data Center GPU Manager (DCGM). Which DCGM feature should they enable to detect and alert on GPU memory errors that could lead to silent data corruption?

Accepted Answer

DCGM health checks with the `memory` watch.. DCGM health checks with the `memory` watch continuously monitor GPU memory for ECC errors and other faults that can cause silent data corruption. This feature can be configured to raise alerts when errors exceed thresholds, enabling proactive remediation. In contrast, diagnostics are run on-demand, utilization metrics reflect workload, and AutoBoost affects clocks. For detecting memory errors that could corrupt LLM outputs, the memory watch is the correct DCGM feature to enable.

Answer

DCGM configuration with `EnableAutoBoost` set to true.

Answer

DCGM profiling metrics with `DCGM_FI_DEV_GPU_UTIL`.

Answer

DCGM diagnostics with the `-r` option for a full run.

Question 13

A team is deploying a 13B-parameter chatbot on a single NVIDIA A10G GPU (24 GB VRAM). The model's weights are stored in FP16, and the runtime runs out of memory during KV cache allocation under concurrent user sessions. They must keep answer quality essentially unchanged while maximizing concurrent sessions. Which optimization should they apply first?

Accepted Answer

Apply INT8 weight-only quantization to the linear layers with NVIDIA TensorRT-LLM, keeping activations in FP16.. The bottleneck is persistent weight memory (FP16 weights dominate VRAM), which leaves too little room for paged KV cache blocks under concurrency. Weight-only INT8 quantization in TensorRT-LLM roughly halves the linear-layer weight footprint while keeping activations in FP16, preserving quality. That reclaimed VRAM goes directly to KV cache, raising the number of simultaneous sessions without retraining or a new GPU.

Answer

Convert the checkpoint to TF32 and rebuild the TensorRT-LLM engine with builder optimization level 5.

Answer

Increase the paged KV cache block size from 16 to 128 tokens per block and disable block reuse.

Answer

Enable CUDA graph capture for the decoder and set the maximum batch size to the peak observed concurrency.

Question 14

An engineer is using TensorRT-LLM to serve a chatbot model. They observe that the time to first token (TTFT) is high, but subsequent tokens are generated quickly. Which optimization should they prioritize to reduce TTFT?

Accepted Answer

Optimize the prefill phase with kernel fusion and faster attention.. Time to first token is primarily determined by the prefill phase, which processes the input prompt. Optimizing this phase with fused kernels, efficient attention mechanisms, and reduced memory latency directly reduces TTFT. TensorRT-LLM includes highly optimized prefill kernels and supports FlashAttention, which significantly speeds up the prefill computation. Other options target throughput or cache management, not TTFT.

Answer

Enable in-flight batching to increase GPU utilization.

Answer

Increase the KV cache size to avoid evictions.

Answer

Use a larger batch size for the prefill phase.

Question 15

An engineer is using NVIDIA TensorRT to optimize a Transformer model for inference on an NVIDIA A100 GPU. They want to maximize throughput while ensuring that the model runs correctly with varying input sequence lengths. Which TensorRT feature should they configure to allow the engine to handle different input shapes at runtime?

Accepted Answer

Dynamic shapes with optimization profiles. Dynamic shapes with optimization profiles enable a single TensorRT engine to handle inputs of different sizes by defining shape ranges. This allows the engine to optimize for the actual input shape at runtime, which is crucial for Transformer models with variable sequence lengths. Static shapes, multiple engines, or INT8 calibration do not provide this flexibility.

Answer

Static shapes with fixed dimensions

Answer

INT8 calibration with a representative dataset

Answer

Multiple engines for each possible input length