NCP-GENL GPU Acceleration and Optimization Practice Test 3 — 15 Questions

Question 1

You are optimizing a ResNet-50 model on an NVIDIA A100. Which precision-based optimization will yield the highest throughput without significant accuracy loss?

Accepted Answer

Mixed-precision (FP16/BF16).. NVIDIA A100 GPUs feature Tensor Cores that are specifically optimized for FP16 and BF16 arithmetic. Utilizing mixed-precision training or inference allows the model to leverage these high-speed units, effectively doubling throughput compared to FP32. This is the standard approach for balancing the trade-off between floating-point precision and computational speed, providing near-native accuracy while maximizing the hardware's performance capabilities in deep learning workloads.

Answer

Full FP32 precision.

Answer

Aggressive FP8 quantization.

Answer

Custom boolean quantization.

Question 2

Which NVIDIA library is primarily used for optimizing and deploying deep learning inference models?

Accepted Answer

TensorRT.. TensorRT is NVIDIA's dedicated deep learning inference optimizer and runtime engine. It takes pre-trained models from frameworks like PyTorch or TensorFlow and optimizes them for NVIDIA hardware by fusing layers, selecting best-fit kernels, and performing precision calibration. It is the core tool for any developer needing to transition models from research training to production-level deployment with minimal latency and maximal throughput on NVIDIA infrastructure.

Answer

cuBLAS.

Answer

NCCL.

Answer

cuDNN.

Question 3

A team is training a large language model on 8 NVIDIA A100 GPUs using PyTorch's DistributedDataParallel (DDP). Profiling shows that all GPUs are frequently idle, waiting for gradient synchronization. The network interconnect between nodes is a 100 Gb Ethernet with TCP/IP, and the model has 13 billion parameters. What is the most effective optimization to reduce the idle time?

Accepted Answer

Replace the Ethernet interconnect with NVIDIA Mellanox InfiniBand and enable NCCL over RDMA.. The idle time is caused by slow gradient synchronization over TCP/IP Ethernet. InfiniBand with RDMA and NCCL provides the necessary bandwidth and low latency to reduce all-reduce time, directly addressing the bottleneck. Other options either do not target communication or are secondary optimizations that do not resolve the fundamental interconnect limitation.

Answer

Increase the batch size per GPU to keep the GPUs busy during communication.

Answer

Enable gradient compression using FP16 all-reduce to halve communication volume.

Answer

Use NVIDIA GPUDirect Storage to accelerate data loading from NVMe drives.

Question 4

A team is serving a 13B-parameter LLM on a single NVIDIA A100 80GB GPU. During generation, they observe that the GPU compute utilization stays below 20% while memory bandwidth utilization is near saturation. They want to improve throughput without changing the model architecture. Which optimization is most appropriate?

Accepted Answer

Apply 4-bit weight-only quantization to reduce memory traffic during token generation.. The symptoms indicate a memory-bandwidth-bound workload, common in autoregressive LLM decoding. Reducing weight precision via 4-bit quantization decreases the volume of data read from GPU memory per token, directly increasing throughput. Other options either do not target memory bandwidth or introduce new bottlenecks.

Answer

Enable multi-GPU tensor parallelism across two A100 GPUs to split the model.

Answer

Increase the batch size to improve arithmetic intensity and better utilize tensor cores.

Answer

Increase the number of CPU threads used for token sampling to speed up generation.

Question 5

An engineer is deploying a large language model using NVIDIA TensorRT-LLM on an A100 GPU. The model uses multi-head attention with a sequence length of 4096. During inference, the GPU's Tensor Cores are underutilized, and the kernel launch overhead is high due to many small operations. Which optimization should be applied to improve Tensor Core utilization and reduce overhead?

Accepted Answer

Enable kernel fusion by using TensorRT-LLM's fused multi-head attention plugin.. The underutilization of Tensor Cores and high kernel launch overhead stem from many small operations in multi-head attention. TensorRT-LLM's fused multi-head attention plugin combines these operations into a single optimized kernel, reducing overhead and improving Tensor Core utilization. This is a targeted optimization for Transformer models, especially with long sequences.

Answer

Increase the batch size to maximize parallelism and hide latency.

Answer

Use NVIDIA Triton Inference Server with dynamic batching to improve GPU utilization.

Answer

Convert the model to FP16 precision to double the Tensor Core throughput.

Question 6

A team is training a 13B-parameter LLM on 8 NVIDIA A100 GPUs using NVIDIA NeMo. They observe that the all-reduce communication during data-parallel training consumes nearly 40% of each iteration. Which of the following changes is most likely to reduce this communication overhead while preserving convergence?

Accepted Answer

Use NVIDIA NCCL with the ring algorithm and overlap communication with computation via gradient bucketing.. The communication bottleneck in data-parallel training is best mitigated by using an efficient collective algorithm and hiding latency. NCCL's ring algorithm excels with large messages on high-bandwidth links, and gradient bucketing enables overlap with compute. This preserves the data-parallel semantics and convergence while reducing wall-clock time spent in all-reduce.

Answer

Switch from data parallelism to tensor parallelism across all 8 GPUs.

Answer

Enable gradient accumulation with a larger micro-batch size and use NCCL with tree algorithm.

Answer

Reduce the number of GPUs to 4 and increase the per-GPU batch size.

Question 7

A team is training a large language model using NVIDIA DGX A100 nodes with 8 GPUs per node. They observe that GPU utilization is high on all GPUs, but the training throughput scales poorly when adding more nodes. Profiling shows that the communication time during all-reduce operations increases significantly with node count. Which of the following optimizations is most likely to improve scaling efficiency?

Accepted Answer

Configure NCCL to use InfiniBand with GPUDirect RDMA and enable adaptive routing.. The poor scaling with increasing node count points to inter-node communication overhead, specifically during all-reduce operations. Leveraging InfiniBand with GPUDirect RDMA and adaptive routing optimizes the network path, reducing latency and improving bandwidth. This directly targets the communication bottleneck, enabling better scaling efficiency for distributed LLM training across multiple DGX nodes.

Answer

Use gradient accumulation to increase the effective batch size.

Answer

Increase the number of CPU threads dedicated to data loading.

Answer

Enable NCCL's tree algorithm for all-reduce operations.

Question 8

A developer is using NVIDIA TensorRT to optimize a BERT-based model for inference. They notice that the engine performs poorly on variable-length input sequences because it was built with a single optimization profile for a fixed sequence length. What should they do to improve performance across different sequence lengths?

Accepted Answer

Rebuild the engine with multiple optimization profiles covering the range of sequence lengths.. TensorRT optimization profiles allow the engine to tune kernels for specific input shape ranges. When input lengths vary, a single profile cannot cover all cases efficiently. Creating multiple profiles, each with appropriate min/opt/max dimensions, enables the engine to choose the best kernels for each length, improving overall performance.

Answer

Pad all input sequences to the maximum length supported by the model.

Answer

Use CUDA graphs to capture and replay the inference execution.

Answer

Enable FP16 precision to reduce memory usage and speed up kernels.

Question 9

A developer is optimizing a generative AI model for inference on NVIDIA GPUs. They want to reduce memory footprint and improve throughput without sacrificing accuracy. Which two techniques should they apply? (Choose two.)

Accepted Answer

Use NVIDIA TensorRT with INT8 quantization and calibration.. INT8 quantization and structured sparsity both reduce memory footprint and improve throughput. INT8 quantization uses 8-bit integers, cutting memory usage by 4x compared to FP32, and Tensor Cores accelerate INT8 operations. Structured sparsity prunes weights in a 2:4 pattern, halving memory for weights and enabling sparse Tensor Core acceleration. Both are supported by TensorRT and maintain accuracy with proper calibration and fine-tuning.

Answer

Use NVIDIA GPUDirect Storage to offload model weights to NVMe.

Answer

Increase the batch size to amortize memory overhead.

Answer

Enable NVIDIA Ampere TF32 precision for matrix multiplications.

Question 10

An engineer is deploying a large language model using NVIDIA TensorRT-LLM on an A100 GPU. They want to maximize throughput for a chatbot workload with variable-length inputs and outputs. Which of the following techniques should they implement to achieve the highest throughput while maintaining acceptable latency?

Accepted Answer

Enable in-flight batching (continuous batching) and use paged KV cache with block-based memory management.. In-flight batching and paged KV cache are key optimizations in TensorRT-LLM for LLM serving. In-flight batching allows sequences to join and leave the batch at each decoding step, avoiding padding and keeping the GPU busy. Paged KV cache manages memory in blocks, enabling more sequences to run concurrently. This combination delivers the highest throughput for variable-length chatbot workloads.

Answer

Use static batching with a fixed batch size of 32 and pad all sequences to the maximum length.

Answer

Use FP32 precision to ensure accuracy and rely on CUDA graphs to reduce launch overhead.

Answer

Increase the number of GPUs and use tensor parallelism with a fixed batch size per GPU.

Question 11

An engineer is optimizing a Transformer-based LLM for inference on an NVIDIA A100 GPU. The model uses FP16 precision, but during generation, the GPU's Tensor Cores are underutilized, and latency is higher than expected. Profiling reveals that many small matrix multiplications are executed sequentially. Which technique is most effective to improve Tensor Core utilization and reduce latency?

Accepted Answer

Apply kernel fusion to combine multiple small operations into a single kernel.. The underutilization of Tensor Cores stems from many small, sequential matrix multiplications. Kernel fusion combines these into larger operations, allowing Tensor Cores to process more data per instruction and reducing kernel launch overhead. This directly improves utilization and lowers latency, making it the most effective technique among the options for this specific bottleneck.

Answer

Enable CUDA graphs to capture and replay the sequence of kernel launches.

Answer

Use NVIDIA TensorRT to quantize the model to INT8.

Answer

Increase the batch size to amortize kernel launch overhead.

Question 12

An engineer is training a large language model with pipeline parallelism across four NVIDIA GPUs. They observe that GPU utilization is low and training throughput is limited by idle time during pipeline bubbles. Which technique is most effective to reduce pipeline bubbles and improve utilization?

Accepted Answer

Increase the number of micro-batches to keep the pipeline stages busy.. Pipeline bubbles are idle periods when stages wait for data. Increasing the number of micro-batches allows more fine-grained overlap of computation across stages, keeping all GPUs busy and reducing idle time. This is a direct and effective way to improve pipeline utilization.

Answer

Reduce the batch size to decrease the memory footprint and allow more concurrent kernels.

Answer

Enable gradient checkpointing to trade compute for memory and increase batch size.

Answer

Switch from pipeline parallelism to data parallelism across the four GPUs.

Question 13

A developer is optimizing a BERT-based model for inference on an NVIDIA T4 GPU using TensorRT. The model has a fixed input sequence length of 128. Profiling shows that the kernel execution time is high due to many small operations. Which TensorRT feature should they use to reduce kernel launch overhead and improve latency?

Accepted Answer

Apply CUDA graphs to capture the entire inference graph and replay it with a single launch.. CUDA graphs are designed to reduce kernel launch overhead by capturing a static sequence of operations and replaying it as a single unit. For a fixed-shape model like the BERT model with sequence length 128, the graph can be captured once and reused across inferences. This eliminates the CPU-side launch latency for each kernel, directly improving latency.

Answer

Increase the batch size to amortize kernel launch overhead across more samples.

Answer

Use TensorRT's builder optimization level 5 to enable aggressive layer fusion.

Answer

Enable FP16 precision and calibrate with a representative dataset.

Question 14

A team is training a large language model on a single NVIDIA H100 GPU. They observe that training throughput is significantly lower than expected, and profiling with Nsight Systems shows long periods where the GPU is idle waiting for data. The data loading pipeline uses the default PyTorch DataLoader with num_workers=0 and no pinned memory. Which change is most likely to improve GPU utilization?

Accepted Answer

Enable pinned memory and increase num_workers in the DataLoader.. The profiling evidence points to a data-loading bottleneck: the GPU sits idle waiting for batches. Enabling pinned memory and increasing DataLoader workers allows asynchronous, overlapped data transfer and prefetching, keeping the GPU fed. The other options target compute or launch overhead, which are not the limiting factors in this scenario.

Answer

Use CUDA graphs to capture the training step and reduce kernel launch overhead.

Answer

Switch from FP32 to TF32 precision for matrix multiplications.

Answer

Increase the batch size to better saturate the GPU.

Question 15

An engineer is optimizing a large language model for inference on NVIDIA GPUs and wants to reduce memory usage to fit a larger model or increase batch size. Which two techniques are most effective for reducing GPU memory consumption during inference? (Choose two.)

Accepted Answer

Quantize model weights to 8-bit integers using TensorRT or similar tools.. Quantizing weights to INT8 reduces memory footprint by storing weights in lower precision, while paged attention optimizes KV cache memory by reducing fragmentation. Both are effective for fitting larger models or increasing batch size during inference. Activation checkpointing and micro-batching are training or throughput techniques, and CPU offloading introduces latency.

Answer

Enable activation checkpointing to recompute activations during the backward pass.

Answer

Increase the number of micro-batches to overlap computation and memory transfers.

Answer

Store the model weights on the CPU and transfer them to the GPU on demand.