NCP-GENL Practice Test 19 — 15 Questions

Question 1

A team is using NVIDIA NeMo Evaluator to assess a Llama-3-70B model fine-tuned for medical question answering. They want to evaluate both the correctness of answers and the model's ability to avoid hallucinating unsupported facts. Which two evaluation strategies should they implement? (Choose two.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use a fact-checking module that cross-references generated statements with a trusted medical knowledge base.. To evaluate correctness, exact match and F1 score against gold answers provide a direct comparison. To detect hallucinations, a fact-checking module that verifies statements against a trusted medical knowledge base is essential. Together, these strategies cover both aspects. BLEU and perplexity do not measure factual accuracy, and a blanket guardrail against numbers is ineffective.

Answer

Compute BLEU score on the generated answers.

Answer

Calculate perplexity of the model on a held-out set of medical questions.

Answer

Run the model through NeMo Guardrails with a policy that blocks any output containing numbers.

Question 2

An LLM inference service on NVIDIA Triton Inference Server is experiencing intermittent failures. The operations team wants to set up alerting to detect when the GPU memory utilization exceeds 90% for more than 5 minutes, as this could lead to out-of-memory errors. Which combination of tools should they use to achieve this?

Accepted Answer

NVIDIA DCGM-Exporter and Prometheus Alertmanager. DCGM-Exporter collects GPU metrics and exposes them to Prometheus. Prometheus evaluates alerting rules based on thresholds and durations, and Alertmanager sends notifications. This setup is ideal for detecting high GPU memory utilization over time, helping prevent OOM errors in LLM inference.

Answer

nvidia-smi with custom scripting and email notifications

Answer

NVIDIA Nsight Systems and Prometheus

Answer

NVIDIA Triton Inference Server metrics and Grafana

Question 3

A generative AI application deployed on NVIDIA Triton Inference Server with NVIDIA AI Enterprise is experiencing silent data corruption in model outputs. The MLOps team needs to implement monitoring to detect such issues early. Which two actions should be taken to enhance observability for silent data corruption? (Choose two.)

Accepted Answer

Enable NVIDIA Data Center GPU Manager (DCGM) to monitor GPU ECC error counts and XID errors.. Silent data corruption can stem from hardware faults or software bugs. Monitoring GPU ECC and XID errors via DCGM catches hardware-induced corruption, while output validation against a baseline detects incorrect results regardless of cause. Together, they provide both infrastructure and application-level observability, enabling early detection and mitigation of silent data corruption in production LLM inference.

Answer

Configure Triton's model warmup to run dummy inferences at startup to stabilize performance.

Answer

Increase the batch size for all inference requests to improve throughput and reduce per-request overhead.

Answer

Set up logging of all inference inputs and outputs to a centralized system for later analysis.

Question 4

You are deploying a large language model on NVIDIA Triton Inference Server in a Kubernetes cluster. To ensure high availability and reliability, which TWO practices should you implement? (Choose two.)

Accepted Answer

Set up health checks and readiness probes for the Triton server pods.. High availability in Kubernetes requires redundancy and health monitoring. Deploying multiple replicas across nodes ensures that a single node failure does not take down the service. Readiness probes ensure traffic is only sent to healthy pods. Together, these practices minimize downtime and maintain reliable inference serving.

Answer

Enable Triton's model warmup to reduce cold-start latency.

Answer

Use a single large GPU instance to maximize performance and reduce complexity.

Answer

Configure multiple model instances per GPU to increase concurrency.

Question 5

A healthcare analytics company is deploying an LLM-based patient triage assistant using NVIDIA NIM microservices. Compliance requires that every model response be traceable to a specific model version, input prompt, and retrieved context for a minimum of three years. Which approach best satisfies this auditability requirement?

Accepted Answer

Instrument the NIM inference pipeline to emit structured audit records containing model ID, prompt hash, retrieved context, and response to a write-once log store.. Traceability for regulated LLM deployments requires capturing the full inference lineage at the moment of generation. Structured, immutable audit records that bind model version, prompt, retrieved context, and response give auditors a reproducible chain of evidence. Guardrail logs, GPU telemetry, and cold-stored weights each cover only part of the picture and cannot substitute for per-inference provenance.

Answer

Enable NeMo Guardrails output moderation and rely on the guardrail logs to reconstruct decisions.

Answer

Increase the model's temperature logging verbosity and store raw GPU telemetry alongside responses.

Answer

Configure the NIM container to retain its model weights and prompt templates for three years in cold storage.

Question 6

An evaluation team is comparing two candidate checkpoints of the same fine-tuned model on a 400-prompt open-ended task. They use an LLM-as-judge that returns a pairwise preference for each prompt. Candidate X wins 214 times, candidate Y wins 152 times, and 34 are ties. The judge is the same family as the models being compared. What should the team do before treating candidate X as the winner?

Accepted Answer

Verify the judge's reliability with a human-labeled subset, control for position bias by swapping answer order, and check that the win margin exceeds judge noise.. Pairwise LLM judging is a measurement instrument that needs validation before its verdicts are trusted. Swapping answer order neutralizes position bias, a human-labeled subset estimates judge accuracy, and comparing the win margin to that measured noise tells the team whether the difference is real. Raw majorities, a switch to overlap metrics, and simply enlarging the sample all fail to address systematic judge bias.

Answer

Discard the judge and rerun the comparison with ROUGE-L, since automatic overlap metrics are objective and free of style bias.

Answer

Accept the result because 214 wins out of 366 non-tie judgments is a clear majority and the sample is large enough to be conclusive.

Answer

Increase the prompt count to 4,000 and keep the same judging procedure, because a tenfold larger sample will average out any judge bias.

Question 7

An engineer is evaluating a sparse Mixture-of-Experts decoder-only LLM for a latency-sensitive inference service. They notice that although the model has far more total parameters than a dense baseline, throughput per token is only modestly better and sometimes worse. Which factor best explains why sparse MoE does not translate total parameter count into proportional speedup during inference?

Accepted Answer

Only the top-k experts are activated per token, so memory bandwidth and all-to-all communication dominate rather than raw FLOPs.. Sparse MoE activates only a few experts per token, so FLOPs are low, but every expert's weights must remain resident and be routed to, often across devices. Inference becomes bound by memory bandwidth and all-to-all communication rather than compute, so a larger total parameter count does not yield proportional latency gains and can even hurt if routing is imbalanced.

Answer

Expert parameters are stored in FP32 while dense models use FP16, doubling memory traffic.

Answer

The router must compute a full softmax over all experts and backpropagate through every expert for each token.

Answer

The attention layers in MoE models are replaced by expert routing, removing the quadratic attention cost.

Question 8

A media-analytics firm serves a 13B-parameter summarization model on two A100 GPUs using NVIDIA TensorRT-LLM behind Triton Inference Server. Traffic is bursty: during live events concurrency triples for about ten minutes, then returns to baseline. Operators report that the first requests after each burst begin are slow and sometimes time out, although steady-state latency is acceptable. Which deployment change most directly addresses the cold-start penalty at the beginning of each burst?

Accepted Answer

Configure Triton's instance groups to keep a resident model instance and enable the model's warmup configuration so activation buffers and CUDA graphs are exercised before live traffic arrives.. The reported pattern, acceptable steady-state latency but slow and timing-out requests at the start of each burst, is a classic cold-start symptom. Keeping a resident Triton instance and supplying a warmup configuration forces engine loading, workspace allocation, and CUDA graph capture to happen at server startup rather than on the first live request. This removes the initialization stall precisely when traffic spikes.

Answer

Switch the Triton scheduler from dynamic batching to sequence batching so that in-flight conversations are tracked per client.

Answer

Enable TensorRT-LLM's in-flight batching and raise the KV cache fraction so more concurrent sequences can share the cache.

Answer

Increase the TensorRT-LLM engine's max_batch_size so larger batches can be formed during the burst peaks.

Question 9

You are evaluating a generative AI model for a chatbot that must adhere to strict safety guidelines. The model occasionally generates toxic or biased responses. You need to automatically evaluate the model's outputs for toxicity and bias before deployment. Which NVIDIA tool or framework is specifically designed for this purpose?

Accepted Answer

NVIDIA NeMo Guardrails. NeMo Guardrails is designed to enforce safety and content policies in conversational AI, including toxicity and bias detection. The other options are infrastructure or optimization tools without safety evaluation features. Thus, NeMo Guardrails is the correct choice for automatically evaluating and mitigating toxic or biased responses before deployment.

Answer

NVIDIA Triton Inference Server

Answer

NVIDIA DALI

Answer

NVIDIA TensorRT

Question 10

An LLM inference service on NVIDIA Triton Inference Server is experiencing a gradual increase in P99 latency over several hours without a corresponding increase in request rate. GPU utilization remains stable at around 60%. Which action should the team take first to diagnose the root cause?

Accepted Answer

Check for memory leaks in the model or Triton server process.. The gradual increase in P99 latency with stable GPU utilization and request rate suggests a resource leak, such as memory fragmentation or a leak in the model or Triton process. Checking for memory leaks is the most direct first step to diagnose the issue. Scaling or profiling may be premature and could mask the root cause. Reviewing input data is unlikely given the stable utilization.

Answer

Increase the number of model instances to handle the load.

Answer

Profile the inference pipeline with NVIDIA Nsight Systems to identify bottlenecks.

Answer

Review the model's input data for changes in sequence length.

Question 11

An LLM inference service running on NVIDIA Triton Inference Server is configured with model ensembles. During production monitoring, the operations team notices that the end-to-end latency reported by the client is significantly higher than the sum of individual model latencies reported by Triton's metrics. Which Triton feature should be investigated to identify the source of the additional latency?

Accepted Answer

Model ensemble scheduler overhead and inter-model data transfer. In Triton model ensembles, the ensemble scheduler coordinates execution and data transfer between models. This adds overhead that is not attributed to any single model's latency. Monitoring ensemble-specific metrics and tracing the pipeline can identify where the extra time is spent, such as in data serialization or scheduling delays.

Answer

GPU memory utilization causing queuing of inference requests

Answer

Network latency between the client and the Triton server

Answer

Dynamic batching configuration causing delays in request aggregation

Question 12

You are evaluating a fine-tuned LLM for a code generation task. The model was trained using NVIDIA NeMo on a dataset of Python functions. You want to measure the percentage of generated functions that pass a set of unit tests. Which evaluation metric is most appropriate?

Accepted Answer

pass@k. The goal is to measure functional correctness by running unit tests. pass@k does exactly that by executing generated code and checking test outcomes, while also accounting for multiple samples. BLEU, ROUGE-L, and perplexity are text-based metrics that do not verify execution or test passage, so they cannot reliably assess code generation success in this scenario.

Answer

ROUGE-L

Answer

BLEU score

Answer

Perplexity

Question 13

A team is fine-tuning a pretrained decoder-only model on a small domain-specific dataset. They observe that the model quickly overfits and loses general language ability. They want to update only a small number of additional parameters while keeping the base weights frozen. Which approach should they use?

Accepted Answer

Low-Rank Adaptation (LoRA), which injects trainable low-rank matrices into existing layers while freezing the base weights.. Parameter-efficient fine-tuning methods such as LoRA freeze the pretrained weights and train only small injected matrices, which reduces optimizer memory and limits drift from the base model. This is well suited to small domain datasets where full fine-tuning would overfit and degrade general language ability.

Answer

Full fine-tuning with a very low learning rate and early stopping.

Answer

Increasing the dropout rate in the feed-forward layers during fine-tuning.

Answer

Freezing the embedding layer only and training all transformer blocks.

Question 14

A team is training a large language model and wants to reduce the memory used by the optimizer without changing the model architecture. They are using Adam and notice that optimizer state consumes more GPU memory than the model weights. Which technique should they apply to reduce optimizer memory while keeping the model architecture unchanged?

Accepted Answer

Switch to a memory-efficient optimizer such as 8-bit Adam or Adafactor that stores reduced-precision or factored optimizer state.. Adam maintains two full-precision moment estimates for every trainable parameter, so optimizer state can be several times the size of the model weights. Memory-efficient optimizers such as 8-bit Adam or Adafactor reduce this footprint by quantizing or factoring the state, directly lowering the dominant memory cost without changing the model architecture.

Answer

Replace the feed-forward activation with a gated variant to reduce activation memory.

Answer

Reduce the number of attention heads while keeping the hidden size constant.

Answer

Increase the batch size proportionally to the number of GPUs so optimizer states are shared across more tokens.

Question 15

An engineer is reviewing the attention implementation of a decoder-only LLM used for chat. During inference with a KV cache, generated tokens must not attend to future positions. Which mechanism enforces this constraint inside scaled dot-product attention?

Accepted Answer

A causal mask that sets attention scores for future positions to negative infinity before the softmax.. Causal masking is the standard mechanism that makes decoder-only attention autoregressive. By adding negative infinity to scores of future positions before softmax, those positions receive zero weight, so each token can only attend to itself and prior tokens. Other listed techniques affect numerical stability or positional encoding but do not enforce the temporal constraint.

Answer

Dropping the scaling factor 1/sqrt(d_k) so that scores decay for distant tokens.

Answer

Applying LayerNorm to the query and key projections before computing attention scores.

Answer

Shifting the position IDs of cached keys so they appear before the current token.