NCP-GENL Practice Test 20 — 15 Questions

Question 1

A team is deploying a large language model on NVIDIA Triton Inference Server in a production environment. They need to ensure that the model server can automatically recover from GPU failures without manual intervention. Which feature of Triton should they configure to achieve this?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Triton's health check endpoints and Kubernetes liveness probes. To automatically recover from GPU failures, Triton should be deployed in an orchestrated environment like Kubernetes. Triton provides health check endpoints that, when integrated with Kubernetes liveness probes, allow the orchestrator to detect an unhealthy server and restart it, possibly on a different node with a working GPU. This approach ensures minimal downtime without manual intervention.

Answer

Instance groups with multiple GPU instances

Answer

Model repository polling

Answer

Model ensembles

Question 2

A production LLM service on NVIDIA Triton Inference Server is deployed across multiple GPUs. The team notices that one GPU consistently shows higher latency for inference requests compared to others, despite similar utilization. Which NVIDIA tool should be used to investigate per-GPU performance discrepancies and identify bottlenecks?

Accepted Answer

NVIDIA Nsight Systems. Nsight Systems provides detailed profiling of GPU activities, including kernel execution, memory transfers, and synchronization. By capturing a timeline across all GPUs, it can reveal why one GPU lags, such as longer kernel runtimes or thermal issues. This level of detail is essential for diagnosing per-GPU performance discrepancies and optimizing LLM inference in a multi-GPU deployment.

Answer

NVIDIA Triton Inference Server metrics

Answer

NVIDIA Data Center GPU Manager (DCGM)

Answer

NVIDIA System Management Interface (nvidia-smi)

Question 3

A team is deploying a 70B-parameter decoder-only LLM on an NVIDIA H100 GPU node. During generation they observe that the KV cache grows linearly with sequence length and is consuming most of the available HBM, forcing them to limit batch size. They want to reduce KV cache memory without retraining the model from scratch. Which architectural change should they apply?

Accepted Answer

Replace multi-head attention with Grouped Query Attention (GQA), where multiple query heads share a single key/value head.. Grouped Query Attention reduces the number of key/value heads relative to query heads, so the KV cache that must be stored for each token shrinks by the grouping factor. Because the cache dominates HBM during long-context decoding, this architectural change directly relieves memory pressure and permits larger batches, and it can be adopted via uptraining rather than a full retrain.

Answer

Switch the feed-forward network activation from SwiGLU to ReLU to reduce parameter count.

Answer

Apply rotary positional embeddings to the key and query vectors instead of learned absolute embeddings.

Answer

Increase the number of attention heads while keeping the head dimension constant.

Question 4

A developer is inspecting a decoder-only Transformer and notices that during training, the model attends to future tokens in the sequence, causing the loss to drop unrealistically fast but generation to be incoherent. Which architectural mechanism is missing or misconfigured?

Accepted Answer

Causal attention mask in the self-attention layers. Autoregressive language models require a causal mask that sets attention scores to negative infinity for positions after the current token. When that mask is absent, the training objective becomes trivial because the target token is visible in the input context, producing the fast loss drop and incoherent generation. Positional encodings, dropout, and normalization placement do not enforce temporal directionality.

Answer

Pre-layer normalization placement before attention

Answer

Dropout regularization on attention weights

Answer

Rotary positional embedding frequency base

Question 5

A global bank uses NVIDIA NeMo Guardrails to enforce ethical AI policies in its customer-facing LLM application. The compliance team requires that the system automatically logs all instances where the model attempts to generate financial advice, including the prompt, the blocked response, and the rail that triggered. Which NeMo Guardrails feature should the team enable to capture this audit trail?

Accepted Answer

Colang tracing with a custom logging action. Colang tracing with a custom logging action is the correct approach because it hooks into the guardrails execution flow, allowing the team to capture the exact rail that triggered and the associated prompt and response. Other options are infrastructure components that lack visibility into NeMo Guardrails' policy decisions, so they cannot provide the required audit detail.

Answer

NVIDIA TensorRT-LLM's runtime debugging output

Answer

NVIDIA NIM's built-in request logging

Answer

NVIDIA Triton Inference Server's model ensemble scheduler

Question 6

A team is pre-training a 13B-parameter decoder-only LLM on a cluster of NVIDIA GPUs. They observe that gradient norms spike sharply during the first few hundred steps, destabilizing training. They want to keep the standard post-layer-normalization placement but stabilize early optimization. Which architectural technique should they apply?

Accepted Answer

Initialize all residual output projections with a small scaling factor such as 1/sqrt(2N).. Post-layer-normalization Transformers suffer from residual stream variance that grows with depth, producing large gradients early in training. Scaling residual branch outputs at initialization by a depth-dependent factor keeps variance bounded, directly stabilizing optimization without changing normalization placement or the optimizer schedule. Optimizer tweaks and activation swaps do not address the architectural root cause.

Answer

Apply learning rate warmup over the first several thousand steps.

Answer

Insert RMSNorm immediately after the token embedding layer only.

Answer

Replace GELU activations in the feed-forward network with ReLU.

Question 7

An ML engineer is evaluating Mixture-of-Experts (MoE) routing for a large decoder-only model to increase capacity without proportionally increasing compute per token. Which TWO statements accurately describe how top-k token routing behaves in such an architecture? (Choose two.)

Accepted Answer

Routing decisions are computed per token, allowing different tokens in the same sequence to be processed by different expert subsets. Sparse MoE replaces a single feed-forward block with many expert blocks plus a router that, for each token, selects the top-k experts to run. This yields token-level dynamic routing and keeps per-token FLOPs tied to k and expert size rather than the total expert population. Dense averaging, static assignment, and automatic k growth all contradict the sparse, learned, token-specific routing that defines the architecture.

Answer

Increasing the number of experts proportionally increases the number of activated experts per token

Answer

The router assigns tokens to experts once at initialization and keeps the assignment fixed throughout training

Answer

All experts receive every token, but their outputs are averaged with learned scalar weights

Question 8

A hospital's AI governance board is reviewing an LLM triage assistant built on NVIDIA NIM. They want an ongoing, automated mechanism that flags when model outputs drift toward unsafe clinical recommendations across thousands of daily conversations, without reviewing every transcript manually. Which approach best fits this need?

Accepted Answer

Deploy NVIDIA NeMo Guardrails with output rails that classify responses against a clinical-safety policy, and emit metrics to a monitoring dashboard for threshold alerts.. Continuous safety oversight at scale requires machine-evaluable signals rather than manual review or model retraining. Output rails in NeMo Guardrails can classify responses against a clinical policy and emit telemetry, which feeds dashboards and alerts. This gives the governance board an automated, auditable early-warning system for unsafe drift across large volumes of conversations.

Answer

Fine-tune the base model weekly on the most recent transcripts so unsafe recommendations naturally decrease over time.

Answer

Increase the model's temperature setting so the assistant produces more varied responses that are easier to spot during spot checks.

Answer

Require clinicians to sign off on every AI-generated recommendation before it reaches a patient, eliminating the need for automated drift detection.

Question 9

A research team is pretraining a decoder-only LLM and observes that gradient magnitudes in the earliest layers are extremely small while later layers train normally, causing slow convergence. They are using post-layer normalization. Which architectural change is most likely to improve gradient flow to the early layers?

Accepted Answer

Move layer normalization to before each sublayer (pre-LN) and add a final normalization before the output projection. Post-LN places normalization inside the residual branch, so gradients must pass through normalization at every layer, attenuating them in deep stacks. Pre-LN normalizes the sublayer input and leaves the residual stream unnormalized, giving a direct gradient highway from the loss to early parameters. That structural change, plus a final normalization before the output head, is the recognized fix for the described symptom.

Answer

Replace layer normalization with batch normalization across the sequence dimension

Answer

Remove residual connections to shorten the gradient path

Answer

Increase the number of attention heads while keeping head dimension constant

Question 10

An engineer must serve a 70B-parameter LLM for a workload with many concurrent users and long shared system prompts, and wants to maximize throughput without retraining. Which inference-time optimization most directly reduces redundant computation across requests sharing the same prompt prefix?

Accepted Answer

Prefix caching of key/value tensors for shared prompt prefixes. When many requests share a long system prompt, recomputing attention over that prefix for every request wastes prefill compute. Prefix caching persists the key/value tensors for the shared prefix and reuses them, cutting redundant work and improving throughput under concurrency. Speculative decoding targets decode latency, beam search adds work, and KV cache quantization saves memory but not duplicated prefill computation.

Answer

Increasing the beam width during generation

Answer

Speculative decoding with a smaller draft model

Answer

Enabling FP8 quantization of the KV cache

Question 11

An inference engineer is serving a 70B-parameter decoder-only LLM and wants to reduce KV cache memory to fit longer contexts on each GPU. They consider Multi-Query Attention (MQA), Grouped-Query Attention (GQA), and standard Multi-Head Attention (MHA). Which statement correctly describes the memory and quality tradeoff among these attention variants?

Accepted Answer

MQA shares key and value projections across all query heads, giving the smallest KV cache but often degrading quality relative to MHA.. MQA minimizes KV cache by sharing one K and V head across all query heads, at some cost to quality. GQA interpolates by grouping query heads with dedicated K and V heads, balancing memory and quality. MHA provides the richest representation but the largest cache. The other options misstate GQA's structure, reverse MHA's memory ranking, or incorrectly claim GQA removes the cache.

Answer

GQA eliminates the KV cache entirely by recomputing keys and values on the fly during decoding.

Answer

GQA uses a single key and value head shared by all query heads, matching MQA memory savings while preserving MHA quality.

Answer

MHA has the smallest KV cache because each head stores its own keys and values independently.

Question 12

A team is fine-tuning a Llama 3 8B model with NVIDIA NeMo on a single A100 80GB GPU. They observe that validation loss starts to rise while training loss continues to decrease after epoch 2. They want to keep the best generalizing checkpoint without changing the dataset. Which NeMo training configuration strategy should they apply?

Accepted Answer

Enable early stopping based on validation loss and save the best checkpoint using the validation metric.. The divergence between decreasing training loss and increasing validation loss indicates overfitting. The correct response is to use validation-based early stopping and retain the best checkpoint, which NeMo supports through its checkpointing and early stopping mechanisms. Increasing epochs, changing optimizers, or shrinking validation data do not address the core generalization problem.

Answer

Reduce the validation split size so that validation loss more closely tracks training loss.

Answer

Increase the number of training epochs and rely on the final checkpoint for deployment.

Answer

Switch the optimizer from AdamW to SGD with a higher momentum value.

Question 13

An engineer is preparing an ensemble in NVIDIA Triton Inference Server that chains a Python preprocessing model with a TensorRT-LLM backend model. The preprocessing model must run on CPU while the LLM must run on GPU, and the ensemble must expose a single HTTP endpoint. Which configuration is required to make the ensemble execute correctly?

Accepted Answer

Define each step as a separate Triton model and reference them in a platform 'ensemble' model's ensemble_scheduling block. Triton ensembles use a model with platform 'ensemble' and an ensemble_scheduling block that lists each step with model name, version, and tensor mappings. This enables chaining a CPU Python preprocessing model with a GPU TensorRT-LLM model behind a single endpoint, which is exactly the requirement.

Answer

Write a custom backend that loads both models and orchestrates them in C++

Answer

Merge both models into a single TensorRT-LLM engine using a custom plugin

Answer

Use Triton's 'instance_group' to pin both models to the same GPU and rely on default routing

Question 14

An LLM service on NVIDIA Triton Inference Server experiences high time-to-first-token because the dynamic batcher waits for full batches. The team wants to reduce time-to-first-token while still benefiting from batching. Which adjustment is most appropriate?

Accepted Answer

Decrease max_queue_delay_microseconds to shorten batching wait time.. Time-to-first-token is driven by how long the dynamic batcher holds requests before dispatch. Lowering max_queue_delay_microseconds shortens that wait, so requests start sooner while still being batched with any available peers. Raising the delay, inflating preferred batch size, or disabling batching either increases latency or sacrifices throughput unnecessarily.

Answer

Set preferred_batch_size to a very large value.

Answer

Disable dynamic batching entirely for the model.

Answer

Increase max_queue_delay_microseconds to allow larger batches.

Question 15

A retail company wants to let its customer-support LLM answer questions about order status. The security team insists the model must never be able to invoke a refund or account-modification function, even if a user crafts a clever prompt. Which approach enforces that constraint most reliably?

Accepted Answer

Expose only a read-only order-status tool to the model and keep refund and account-modification functions outside the tool registry entirely.. Authorization in tool-using LLM applications must be enforced by what the model can reach, not by what it is told. Keeping refund and account-modification functions out of the tool registry makes them structurally unreachable, so no prompt-injection technique can trigger them. Read-only status access still works. Prompts, fine-tuning, and post-hoc alerting all leave a live execution path that an adversary could exploit.

Answer

Add a system prompt instructing the model to refuse any request that would modify an account or issue a refund.

Answer

Log every tool invocation and alert the security team whenever a refund or account-modification call is detected.

Answer

Fine-tune the model on examples of refund and account-modification refusals so it learns to decline those requests.