NCP-GENL Model Optimization — All Questions With Answers

Question 1mediummultiple choice

Read the full Model Optimization explanation →

An engineer needs to deploy a large language model on resource-constrained edge hardware. Which optimization technique provides the best balance between memory footprint reduction and inference latency?

Question 2mediummulti select

Read the full Model Optimization explanation →

Which TWO of the following TensorRT optimization techniques are most effective for improving throughput in LLMs using the Transformer architecture?

Question 3hardmulti select

Read the full Model Optimization explanation →

Which THREE of the following are benefits of using NVIDIA TensorRT over raw PyTorch for LLM inference in production?

Question 4hardmultiple choice

Read the full Model Optimization explanation →

Refer to the exhibit. The builder fails during engine construction. What is the most likely cause related to the TensorRT optimization pipeline?

Exhibit

Error: [TensorRT] Incompatible dimensions. Input shape [1, 1024] cannot be broadcast to [1, 512, 1024].

Question 5mediummulti select

Read the full Model Optimization explanation →

Which TWO of the following are true regarding the use of 'profiles' in TensorRT for LLMs?

Question 6mediummultiple choice

Read the full Model Optimization explanation →

An engineer needs to deploy a large language model on resource-constrained edge hardware. Which optimization technique provides the highest inference speedup by reducing numerical precision while maintaining model structural integrity?

Question 7hardmulti select

Read the full Model Optimization explanation →

Which TWO of the following NVIDIA TensorRT features are primarily responsible for optimizing Transformer-based model performance during the build phase?

Question 8easymultiple choice

Read the full Model Optimization explanation →

Which optimization strategy is most effective for reducing the inference latency of an LLM when the model exceeds the memory capacity of a single NVIDIA GPU?

Question 9mediummulti select

Read the full Model Optimization explanation →

Which THREE techniques are commonly used to optimize inference throughput in Generative AI applications deployed on NVIDIA Triton Inference Server?

Question 10hardmultiple choice

Read the full Model Optimization explanation →

Which NVIDIA technology provides the most significant boost for LLM inference by utilizing specialized hardware to accelerate non-matrix operations like Softmax and LayerNorm?

Question 11hardmultiple choice

Read the full Model Optimization explanation →

What is the primary benefit of using FP8 precision for inference in modern Transformer models on NVIDIA H100 GPUs?

Question 12mediummultiple choice

Read the full Model Optimization explanation →

An engineer needs to deploy a large language model on NVIDIA H100 GPUs while maximizing throughput for a high-concurrency inference service. Which optimization technique is most effective for reducing latency by minimizing memory bandwidth bottlenecks?

Question 13mediummultiple choice

Read the full Model Optimization explanation →

Which NVIDIA TensorRT-LLM feature is primarily used to optimize the parallel execution of the KV cache across multiple GPU instances in a multi-node setup?

Question 14hardmultiple choice

Read the full Model Optimization explanation →

When optimizing a Transformer-based model using NVIDIA TensorRT, which TWO steps are essential to enable the Fusion of layer normalization and activation kernels to improve performance?

Question 15mediummultiple choice

Read the full Model Optimization explanation →

A team wants to reduce the cold-start latency for an LLM served with TensorRT-LLM. They've already built an engine with FP16 precision and paged KV cache. Which TensorRT-LLM runtime feature should they enable to overlap the compilation of GPU kernels with the loading of model weights?

Question 16mediummultiple choice

Read the full Model Optimization explanation →

An engineer is optimizing a 13B-parameter LLM for a single NVIDIA L40S GPU using TensorRT-LLM. The model must retain high accuracy on a domain-specific question-answering benchmark, and the team has a small calibration set of 512 representative prompts. Post-training INT8 quantization causes a noticeable drop in answer quality, particularly for numeric reasoning. Which approach best preserves accuracy while still reducing memory and latency?

Question 17hardmultiple choice

Read the full Model Optimization explanation →

An engineer is optimizing a 70B parameter LLM for inference on a single NVIDIA H100 GPU with 80GB memory. They want to maximize throughput while keeping latency acceptable. Which combination of TensorRT-LLM settings should they use to fit the model and KV cache in memory?

Question 18hardmultiple choice

Read the full Model Optimization explanation →

An engineer profiles a TensorRT-LLM Llama-2-70B engine on 4x H100 GPUs with tensor parallelism and observes that decode-step latency is dominated by inter-GPU communication rather than matmul time. The engine currently uses tensor parallelism degree 4 with no pipeline parallelism. Which change most directly reduces the communication share during decoding?

Question 19mediummultiple choice

Read the full Model Optimization explanation →

A team is serving a 70B-parameter LLM with NVIDIA TensorRT-LLM on two NVIDIA H100 GPUs using tensor parallelism. Under production traffic with many concurrent users, GPU utilization drops to roughly 40% and throughput plateaus well below the target, even though per-token latency stays acceptable. Profiling shows the GPUs idle while waiting on memory transfers between the two GPUs. Which optimization should the team apply first to raise throughput?

Question 20hardmultiple choice

Read the full Model Optimization explanation →

An engineer is building a TensorRT-LLM engine for a 13B chat model on a single A100 80 GB GPU. The workload mixes short prompts with long multi-turn conversations, and the team wants to minimize end-to-end latency for interactive use without retraining. After enabling FP8 or INT8 weight-only quantization, which additional optimization most directly attacks the memory-bandwidth bottleneck during autoregressive decoding?

Question 21mediummultiple choice

Read the full Model Optimization explanation →

An enterprise deploying a large language model on an NVIDIA A100 GPU experiences high memory bandwidth bottlenecks during autoregressive token generation. Which optimization technique specifically addresses this memory-bound phase by merging element-wise operations and reducing global memory round-trips?

Question 22mediummultiple choice

Read the full Model Optimization explanation →

When optimizing a model using NVIDIA TensorRT, what is the primary benefit of enabling 'layer fusion' during the optimization process?

Question 23hardmultiple choice

Read the full Model Optimization explanation →

Refer to the exhibit. The TensorRT build process fails with a memory limit error. Which configuration adjustment is most likely to resolve this build-time error?

Exhibit

Error: [TensorRT] Assertion failed: engine_size < max_device_memory_limit. Plan execution aborted.

Question 24mediummultiple choice

Read the full Model Optimization explanation →

Which strategy is most effective for optimizing an LLM that is too large to fit into a single GPU's VRAM?

Question 25easymultiple choice

Read the full Model Optimization explanation →

What is the primary role of an inference 'calibrator' when converting a model to INT8 precision?

Question 26mediummultiple choice

Read the full Model Optimization explanation →

Which hardware architecture feature is specifically leveraged by TensorRT to accelerate FP16 and INT8 matrix multiplications?

Question 27hardmultiple choice

Read the full Model Optimization explanation →

Refer to the exhibit. The engineer is attempting to deploy on an NVIDIA Orin platform but encounters a runtime error. What is the most likely cause of the failure?

Network Topology

Question 28mediummultiple choice

Read the full Model Optimization explanation →

In the context of model optimization, why is 'graph surgery' sometimes required before building a TensorRT engine?

Question 29mediummultiple choice

Read the full Model Optimization explanation →

What is the primary advantage of using a 'Quantization Aware Training' (QAT) approach over post-training quantization for LLMs?

Question 30mediummultiple choice

Read the full Model Optimization explanation →

Refer to the exhibit. An engineer receives these logs while converting a Transformer model to a TensorRT engine. What is the most appropriate action to resolve this build failure?

Exhibit

TRT_LOG: [E] Error: Failed to find valid tactics for layer 'Attention_Softmax_0'.
TRT_LOG: [E] Error: Workspace memory limit exceeded for layer 'MatMul_QKV_1'.
TRT_LOG: [I] INFO: Attempting to optimize graph with reduced precision...

Question 31hardmultiple choice

Read the full Model Optimization explanation →

When optimizing a Generative AI model using NVIDIA TensorRT-LLM, which component is primarily responsible for managing the KV cache to minimize memory fragmentation?

Question 32mediummultiple choice

Read the full Model Optimization explanation →

Refer to the exhibit. An engineer observes that GPU memory utilization is high, but the GPU is frequently idling. How does the provided Triton configuration optimize the inference pipeline?

Exhibit

config.pbtxt:
instance_group [
  {
    count: 2
    kind: KIND_GPU
    gpus: [0]
  }
]

Question 33mediummultiple choice

Read the full Model Optimization explanation →

An engineer is tasked with optimizing a model that performs poorly due to excessive memory access latency. Which TensorRT optimization strategy specifically targets this issue?

Question 34easymultiple choice

Read the full Model Optimization explanation →

What is the primary function of the 'Triton Model Analyzer' in an optimization workflow?

Question 35mediummulti select

Read the full Model Optimization explanation →

Which THREE factors should be considered when choosing an optimal batch size for LLM inference on NVIDIA GPUs?

Question 36mediummultiple choice

Read the full Model Optimization explanation →

Which optimization technique specifically helps to manage the memory bandwidth bottleneck during the autoregressive decoding phase of an LLM?

Question 37hardmultiple choice

Read the full Model Optimization explanation →

Refer to the exhibit. An engineer notices that the TensorRT engine takes an excessively long time to build. What is the most likely cause, and how can it be mitigated?

Network Topology

Question 38mediummultiple choice

Read the full Model Optimization explanation →

Which optimization method should be prioritized when the model's inference performance is bottlenecked by the CPU-to-GPU data transfer overhead?

Question 39mediummultiple choice

Read the full Model Optimization explanation →

Which THREE of the following are primary benefits of using PagedAttention in NVIDIA TensorRT-LLM deployments?

Question 40hardmultiple choice

Read the full Model Optimization explanation →

Refer to the exhibit. The deployment is facing memory allocation errors during peak load. Based on the error log, what is the most effective configuration change to resolve the issue while keeping the model architecture constant?

Exhibit

Error: [TRT-LLM] KV Cache block allocation failed. Current allocation: 80% capacity. Request rejected to prevent OOM. Optimization status: KV_CACHE_ENABLED=True, PAGED_ATTENTION=False.

Question 41hardmultiple choice

Read the full Model Optimization explanation →

A team is serving a 70B-parameter LLM with TensorRT-LLM in a multi-tenant environment where requests arrive with widely varying prompt lengths and generation lengths. During load testing, they observe that throughput collapses when a long-context request is scheduled alongside many short requests, and GPU memory fragmentation causes intermittent out-of-memory errors even though total free memory appears sufficient. Which TensorRT-LLM runtime configuration change most directly addresses both the throughput collapse and the memory fragmentation?

Question 42mediummultiple choice

Read the full Model Optimization explanation →

A team is deploying a Llama 2 13B model with NVIDIA TensorRT-LLM on a single A100 40GB GPU. They need to serve 32 concurrent requests with a maximum sequence length of 4096 tokens. They observe that the GPU runs out of memory during inference. Which configuration parameter should they adjust to control the maximum GPU memory allocated for the KV cache?

Question 43easymultiple choice

Read the full Model Optimization explanation →

An engineer is optimizing a BERT-like model for inference using NVIDIA TensorRT. They want to reduce latency further by using lower precision without significant accuracy loss. Which TensorRT precision mode should they choose to enable INT8 inference while maintaining accuracy through calibration?

Question 44mediummulti select

Read the full Model Optimization explanation →

An engineer is using NVIDIA TensorRT-LLM to optimize an LLM for inference. They want to reduce the memory footprint of the KV cache during long-context generation. Which TWO techniques are supported by TensorRT-LLM to achieve this? (Choose two.)

Question 45hardmulti select

Read the full Model Optimization explanation →

A developer is using NVIDIA TensorRT-LLM to optimize a GPT-based model for inference. They want to reduce the model's memory footprint and improve throughput without retraining. Which two techniques can be applied during the TensorRT-LLM build process to achieve these goals? (Choose two.)

Question 46mediummultiple choice

Read the full Model Optimization explanation →

An engineer is using TensorRT-LLM to serve a chatbot model. They observe that the time to first token (TTFT) is high, but subsequent tokens are generated quickly. Which optimization should they prioritize to reduce TTFT?

Question 47mediummultiple choice

Read the full Model Optimization explanation →

An engineer is using NVIDIA TensorRT to optimize a Transformer model for inference on an NVIDIA A100 GPU. They want to maximize throughput while ensuring that the model runs correctly with varying input sequence lengths. Which TensorRT feature should they configure to allow the engine to handle different input shapes at runtime?

Question 48mediummultiple choice

Read the full Model Optimization explanation →

A team is deploying a 13B-parameter chatbot on a single NVIDIA A10G GPU (24 GB VRAM). The model's weights are stored in FP16, and the runtime runs out of memory during KV cache allocation under concurrent user sessions. They must keep answer quality essentially unchanged while maximizing concurrent sessions. Which optimization should they apply first?

Question 49hardmultiple choice

Read the full Model Optimization explanation →

A team is deploying a large language model using NVIDIA TensorRT-LLM on a multi-GPU node with NVLink. They want to minimize inter-GPU communication overhead during inference. Which parallelism strategy should they use to achieve this?

Question 50mediummultiple choice

Read the full Model Optimization explanation →

An engineer is deploying a 13B-parameter LLM with TensorRT-LLM on a single NVIDIA A100 40GB GPU. The FP16 engine requires 26GB for weights, but during generation the KV cache grows beyond remaining memory, causing out-of-memory errors. The team wants to maximize concurrent requests without retraining. Which optimization should they apply first?

Question 51mediummulti select

Read the full Model Optimization explanation →

A team is building a TensorRT-LLM engine for a 7B model that must serve both single-turn short prompts and long multi-turn conversations with a shared system prompt. They want to maximize reuse of computation across requests without changing model weights. Which TWO techniques should they enable? (Choose two.)

Question 52easymultiple choice

Read the full Model Optimization explanation →

A team is preparing a Llama-based chatbot for production and wants to reduce GPU memory and latency without retraining. They decide to apply post-training quantization. Which TensorRT-LLM workflow correctly produces an INT8 or FP8 quantized engine from an existing FP16 checkpoint?

Question 53easymultiple choice

Read the full Model Optimization explanation →

A developer wants to reduce the disk and memory footprint of a fine-tuned 70B model before serving it with TensorRT-LLM, and is willing to accept a small, measurable quality drop that they will validate with an evaluation harness. Which approach best matches that requirement?

Question 54hardmultiple choice

Read the full Model Optimization explanation →

An engineer is using NVIDIA TensorRT-LLM's in-flight batching to serve a mix of short and very long prompts. They observe that GPU utilization drops and latency for short requests spikes whenever a long prompt is admitted. Which mechanism should they tune to prevent long sequences from monopolizing the batch?

Question 55hardmultiple choice

Read the full Model Optimization explanation →

An engineer is using TensorRT-LLM to serve a model that occasionally receives prompts far longer than the typical 512 tokens, up to 8K tokens. With the default engine settings, requests near 8K fail with a cache capacity error while short requests succeed. Which configuration change most directly resolves this without rebuilding for a single worst-case shape?

Question 56mediummulti select

Read the full Model Optimization explanation →

An engineer is optimizing an LLM for inference with NVIDIA TensorRT-LLM and wants to reduce both memory footprint and latency without retraining the model. Which two techniques should they apply? (Choose two.)

Question 57hardmultiple choice

Read the full Model Optimization explanation →

A team has built a TensorRT-LLM engine for a 70B model on four NVIDIA H100 GPUs using tensor parallelism. They now need to serve the same model on a single H100 for a development environment, accepting higher latency. What is the most appropriate approach?

Question 58easymultiple choice

Read the full Model Optimization explanation →

A developer has a fine-tuned Llama-family model in Hugging Face format and wants to run it with NVIDIA TensorRT-LLM on an H100. Which artifact must be produced before the runtime can execute the model?

Question 59mediummulti select

Read the full Model Optimization explanation →

An engineer is tuning a TensorRT-LLM deployment of a 7B model for a latency-sensitive API. Profiling shows that time per output token is higher than expected and that many small kernels run back to back with gaps between them. Which TWO changes are most likely to reduce the per-token latency by cutting kernel launch overhead and redundant memory traffic? (Choose two.)

Question 60hardmultiple choice

Read the full Model Optimization explanation →

A team must serve a 70B model on a single 80 GB GPU for an internal assistant with modest concurrency. Full FP16 weights will not fit alongside the KV cache for the target context length. They want to keep accuracy loss minimal and are willing to spend additional build time. Which approach best fits these constraints?

TRT_LOG: [E] Error: Failed to find valid tactics for layer 'Attention_Softmax_0'. TRT_LOG: [E] Error: Workspace memory limit exceeded for layer 'MatMul_QKV_1'. TRT_LOG: [I] INFO: Attempting to optimize graph with reduced precision...