NCP-GENL Practice Test 21 — 15 Questions

Question 1

A team is deploying a 70B-parameter LLM using NVIDIA Triton Inference Server with TensorRT-LLM backend on a node with four A100 80GB GPUs. They observe that during inference, only one GPU is utilized while the others remain idle. They have configured the model with tensor parallelism set to 1. What is the most likely cause of this underutilization?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Tensor parallelism is set to 1, which means the model is not partitioned across GPUs; it runs entirely on one GPU.. Tensor parallelism determines how a model is sharded across multiple GPUs. Setting it to 1 means no sharding, so the entire model runs on a single GPU, leaving others idle. To utilize all four GPUs for a single model instance, tensor parallelism should be set to 4. Other factors like instance groups or batch size do not override this fundamental partitioning setting.

Answer

The Triton model repository is not configured with the correct instance group for multi-GPU execution.

Answer

The model was compiled with a batch size that is too small to trigger multi-GPU execution.

Answer

The KV cache is not enabled, causing the model to fall back to single-GPU execution.

Question 2

An AI engineer is deploying a large language model using NVIDIA Triton Inference Server. They need to ensure that the server can handle multiple concurrent requests efficiently while maintaining low latency. Which Triton feature allows the server to dynamically batch incoming requests to maximize GPU utilization?

Accepted Answer

Dynamic batching. Dynamic batching is a Triton feature that aggregates concurrent inference requests into larger batches on the server side, improving GPU utilization and throughput while managing latency. It is the standard mechanism for handling high concurrency efficiently. Model versioning, instance groups, and ensemble scheduling serve different purposes and do not provide dynamic request batching.

Answer

Instance groups

Answer

Model versioning

Answer

Ensemble scheduling

Question 3

A team is fine-tuning a 70B parameter model with NVIDIA NeMo using LoRA on eight H100 GPUs. They want to reduce GPU memory usage during training while preserving the base model's pretrained knowledge. Which two configuration changes should they apply? (Choose two.)

Accepted Answer

Enable LoRA by setting adapter dimensions and target modules so only adapter weights receive gradients.. LoRA reduces trainable parameters and optimizer state while freezing the base model, and gradient checkpointing reduces activation memory by recomputing activations. Together they lower peak GPU memory during fine-tuning. Increasing micro batch size, using float32, or disabling tensor parallelism all increase memory usage and work against the stated goal.

Answer

Increase the micro batch size to improve GPU utilization and reduce the number of optimizer steps.

Answer

Disable tensor parallelism so each GPU holds a full copy of the model weights.

Answer

Switch from bfloat16 to float32 precision to improve numerical stability during training.

Question 4

An engineer is deploying a LLM using NVIDIA Triton Inference Server with the TensorRT-LLM backend. They need to ensure that the model can handle a sudden surge in requests without increasing latency beyond a specified threshold. They have configured the model with a maximum batch size of 32 and dynamic batching with a preferred batch size of 16. However, during peak load, latency spikes are observed. Which Triton configuration parameter should they adjust to control the maximum time a request waits in the dynamic batching queue before being processed?

Accepted Answer

max_queue_delay_microseconds. The max_queue_delay_microseconds parameter in Triton's dynamic batching configuration specifies the maximum time a request can wait in the queue before being processed. Lowering it reduces latency spikes under surge conditions by preventing requests from waiting too long for a full batch. Other parameters like max_batch_size and preferred_batch_size affect batching but not the wait timeout.

Answer

max_batch_size

Answer

instance_group

Answer

preferred_batch_size

Question 5

A platform team is deploying a 70B-parameter LLM with NVIDIA TensorRT-LLM across four 80 GB H100 GPUs and needs to serve long-context requests efficiently. They are deciding how to combine parallelism and memory techniques in the build and runtime configuration. (Choose two.)

Accepted Answer

Use tensor parallelism across the four GPUs so each GPU holds a shard of every layer's weights. Tensor parallelism shards each layer across the four H100s so a 70B model fits and serves with low latency over NVLink, while a properly sized paged KV cache prevents memory over-reservation for long contexts. Together they address both weight distribution and the dominant dynamic memory consumer in long-context LLM serving.

Answer

Reduce the number of GPUs to one and enable FP8 quantization to fit the model on a single device

Answer

Rely on CUDA Unified Memory to transparently page weights and KV cache between host and device

Answer

Enable pipeline parallelism with a single micro-batch to minimize inter-GPU communication

Question 6

A developer is preparing a supervised fine-tuning dataset for an instruction-tuned LLM using NVIDIA NeMo. The dataset contains prompts and responses, but the model sometimes learns to generate the prompt text as part of the response. Which dataset formatting practice should be applied to prevent this?

Accepted Answer

Use a loss mask that excludes prompt tokens so the model is trained only on response tokens.. Prompt echoing occurs when the loss is computed over both prompt and response tokens. Applying a loss mask that excludes prompt tokens ensures gradients are only derived from response tokens, which teaches the model to generate answers rather than repeat instructions. Duplicating prompts, extending sequence length, or shuffling pairs do not address the loss computation.

Answer

Shuffle the dataset so prompts and responses are randomly paired during training.

Answer

Increase the maximum sequence length so the prompt and response always fit in a single sample.

Answer

Duplicate the prompt in both the input and output fields to reinforce the instruction format.

Question 7

A team is deploying a large language model on NVIDIA Triton Inference Server with TensorRT-LLM backend. They want to reduce GPU memory consumption to fit a larger model on a single GPU without significantly degrading output quality. Which two techniques should they use? (Choose two.)

Accepted Answer

Enable INT8 weight-only quantization. INT8 weight-only quantization reduces weight memory footprint with minimal quality loss, and paged KV cache optimizes memory usage during generation by reducing fragmentation. Together, they enable larger models to fit on a single GPU. FP32 increases memory, larger batch sizes demand more memory, and tensor parallelism requires multiple GPUs, so they do not meet the goal.

Answer

Increase the maximum batch size

Answer

Use FP32 precision for all computations

Answer

Use tensor parallelism across multiple GPUs

Question 8

An inference team runs a TensorRT-LLM model behind NVIDIA Triton Inference Server with the tensorrtllm_backend. They notice that when clients send requests with widely varying prompt lengths, throughput drops and some requests wait much longer than others. They want Triton to group compatible requests dynamically to improve GPU utilization without changing the model build. Which Triton feature should they configure?

Accepted Answer

Dynamic batching with a preferred batch size and max queue delay in the model's config.pbtxt. Triton's dynamic batching groups independent requests arriving within max_queue_delay into a single batch, using preferred_batch_size to target efficient batch shapes. This improves GPU utilization under varying prompt lengths without rebuilding the TensorRT-LLM engine, directly addressing the throughput and latency variance described.

Answer

Model warmup with sample inputs to pre-allocate memory at load time

Answer

Rate limiting with a max queue size to reject excess requests

Answer

Sequence batching with a sequence ID to maintain state across requests

Question 9

A company needs to deploy a generative AI model that will serve prompts containing regulated customer data. Security policy requires that all inference stays on-premises, that the model be quantized to fit existing GPUs, and that no external network calls occur at runtime. Which deployment approach should the engineer choose?

Accepted Answer

Deploy the model with NVIDIA NIM microservices in an air-gapped environment using locally hosted containers from NVIDIA NGC. NVIDIA NIM microservices are distributed as containers that can be deployed on-premises or in air-gapped environments, support quantized model variants, and do not require external network calls at runtime. This satisfies the on-premises, quantized, and no-external-calls requirements simultaneously, unlike hosted or cloud-based options.

Answer

Run the model locally but allow it to fetch updated weights from a vendor endpoint at startup

Answer

Deploy the model in a public cloud region with a private VPC and customer-managed encryption keys

Answer

Use a public hosted API for the model and rely on TLS encryption for data in transit

Question 10

A media company runs an NVIDIA NIM-hosted content assistant that drafts articles from user prompts. Legal has flagged two risks: the model reproducing long verbatim passages from copyrighted training sources, and the model generating defamatory statements about named private individuals. Which two controls best address these specific risks? (Choose two.)

Accepted Answer

Deploy a retrieval-based similarity check that compares generated output against a licensed corpus index and blocks or rewrites spans that exceed a verbatim-overlap threshold.. The two flagged risks require output-side detection tailored to each: a similarity index against licensed works catches verbatim copyright reproduction, and a named-entity plus defamation classifier with human escalation catches risky claims about private individuals. Sampling changes, model swaps, and rate limiting all operate on the wrong layer and leave one or both risks unmitigated. Layered output controls matched to the specific harm are what the legal review requires.

Answer

Enable request-level rate limiting per user account to reduce the volume of generated articles entering the editorial pipeline.

Answer

Raise the model's temperature and top-p values so generations vary more and are less likely to match any single source.

Answer

Restrict the assistant to a smaller parameter model fine-tuned only on the company's own published archive.

Question 11

A healthcare startup is deploying a Mistral 7B model for internal clinical note summarization. They need to serve the model with NVIDIA Triton Inference Server and want to minimize GPU memory footprint during inference. The team plans to use TensorRT-LLM and is choosing a numerical precision for the engine. Which precision should they select to reduce memory usage while maintaining acceptable accuracy for summarization?

Accepted Answer

INT8 or FP8 quantization, because it reduces weight and activation precision further than FP16 while maintaining acceptable accuracy for text summarization.. INT8 or FP8 quantization reduces the bits per weight and activation below FP16, cutting GPU memory usage while keeping accuracy acceptable for summarization. TensorRT-LLM supports quantized engines with calibration or scaling factors, making it the appropriate precision choice when memory footprint is the primary constraint and the task tolerates small accuracy loss.

Answer

FP16, because it halves memory compared to FP32 and is widely supported on NVIDIA data center GPUs.

Answer

FP32, because it uses the least GPU memory and is the default for TensorRT-LLM engines.

Answer

TF32, because it is a Tensor Core mode that automatically compresses weights to one byte per parameter.

Question 12

A research team is fine-tuning a model with NVIDIA NeMo and wants to reduce the risk of catastrophic forgetting of general capabilities while still adapting to a specialized domain. They have a small domain dataset and limited compute. Which fine-tuning approach best balances domain adaptation with retention of pretrained knowledge?

Accepted Answer

Parameter-efficient fine-tuning with LoRA using a modest rank and a low learning rate.. LoRA with a modest rank and low learning rate freezes the base model and trains small adapters, which constrains updates and preserves pretrained knowledge while still adapting to the domain. Full fine-tuning with high learning rates, training from scratch, or freezing all layers and training only a head either cause forgetting, require excessive resources, or fail to adapt the generative model.

Answer

Full-parameter fine-tuning with a high learning rate for many epochs on the domain dataset.

Answer

Freezing all layers and training only the final classification head on the domain dataset.

Answer

Training from scratch on the domain dataset using the same architecture and tokenizer.

Question 13

A team is deploying a 70B-parameter LLM using NVIDIA Triton Inference Server with TensorRT-LLM backend on a node with four A100 80GB GPUs. They observe that during inference, only one GPU is utilized while the others remain idle. They have configured the model with tensor parallelism set to 1. What is the most likely cause of this underutilization?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Enable NVLink and ensure the GPUs are connected via NVSwitch for peer-to-peer communication.. Tensor parallelism relies on frequent all-reduce operations, so the interconnect bandwidth is critical. NVLink with NVSwitch provides the necessary high-speed peer-to-peer communication to minimize overhead. Without it, PCIe becomes the bottleneck. Other options either do not address the communication pattern or are infeasible for a 70B model on four GPUs.

Answer

Reduce the tensor parallel size to 2 and run two independent replicas.

Answer

Increase the number of attention heads to improve parallelism.

Answer

Switch from tensor parallelism to pipeline parallelism to eliminate inter-GPU communication.

Question 14

An engineer is tasked with optimizing a model that performs poorly due to excessive memory access latency. Which TensorRT optimization strategy specifically targets this issue?

Accepted Answer

Layer Fusion. Layer fusion is the primary strategy for reducing memory access latency. By combining multiple kernels into one, the engine keeps data in the high-speed cache of the GPU instead of constantly writing to and reading from slow global VRAM. This minimizes the time spent waiting for data movement, which is usually the dominant bottleneck for modern neural networks.

Answer

Precision Calibration

Answer

Kernel Auto-Tuning

Answer

Weight Pruning

Question 15

A team is using NVIDIA Triton Inference Server to serve multiple LLMs. They want to automatically detect when a model's inference latency degrades beyond acceptable thresholds and trigger an alert. Which Triton feature should they configure?

Accepted Answer

Triton Metrics API with Prometheus and Alertmanager. Triton's Metrics API, when paired with Prometheus and Alertmanager, provides the necessary observability to monitor inference latency in real time and automatically alert when thresholds are exceeded. This integration is a standard practice for production monitoring, enabling proactive incident response. Other options are either static optimization tools or lack alerting functionality.

Answer

Model Analyzer

Answer

Triton's built-in model warmup

Answer

Triton's dynamic batching configuration