NCP-GENL GPU Acceleration and Optimization Practice Test 2 — 15 Questions

Question 1

A developer is fine-tuning a BERT model on an NVIDIA V100 GPU. They notice that the training step time is much longer than expected, and nvidia-smi shows that GPU memory is nearly full. The model uses a batch size of 64 and the maximum sequence length is 512. Which action is most likely to allow a larger batch size without running out of memory?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Enable mixed precision training with NVIDIA Apex or native AMP.. Mixed precision training reduces memory consumption by storing activations and gradients in FP16, which halves their footprint. This frees enough memory to increase the batch size. It also leverages Tensor Cores for faster math, improving step time. Other options either do not increase per-step batch capacity or alter the model's inputs.

Answer

Use gradient accumulation to simulate a larger batch size.

Answer

Use torch.cuda.empty_cache() before each step.

Answer

Reduce the maximum sequence length to 256.

Question 2

A research team is training a Transformer model on a cluster of NVIDIA A100 GPUs using data parallelism with PyTorch DistributedDataParallel (DDP). They observe that scaling efficiency drops significantly when going from 4 to 8 GPUs. Profiling shows high communication overhead in NCCL AllReduce operations. Which optimization is most likely to improve scaling efficiency in this scenario?

Accepted Answer

Use NCCL with hierarchical AllReduce and ensure NVLink/NVSwitch topology awareness.. The scaling drop is due to communication overhead in AllReduce. NCCL's hierarchical AllReduce takes advantage of intra-node NVLink/NVSwitch for fast local reductions, then performs a smaller inter-node reduction. Ensuring NCCL detects the topology enables optimal algorithm selection, reducing the data transferred over slower links and improving scaling efficiency.

Answer

Switch from DDP to model parallelism to reduce communication.

Answer

Increase the batch size per GPU to amortize communication.

Answer

Enable gradient compression using NCCL's built-in compression.

Question 3

When deploying a large language model on NVIDIA H100 GPUs using TensorRT-LLM, which TWO configuration strategies are most effective for improving KV cache efficiency and memory utilization?

Accepted Answer

Enable PagedAttention to reduce fragmentation.. Optimizing the KV cache is critical for scaling generative AI, as it occupies significant VRAM during inference. PagedAttention dynamically manages memory blocks to prevent fragmentation, while inflight batching ensures that continuous requests are processed without waiting for the entire batch to finish. Combining these allows for higher concurrency and reduced memory overhead, enabling more efficient deployment of models with large context windows on limited GPU hardware.

Answer

Increase the static sequence length allocation.

Answer

Disable multi-head attention optimizations.

Answer

Switch to FP64 precision for all tensors.

Question 4

In the context of NVIDIA Tensor Cores, what is the primary benefit of using BF16 (Bfloat16) over FP16 during model training and inference?

Accepted Answer

BF16 offers a larger dynamic range for gradients.. BF16 uses the same exponent range as FP32, which prevents overflow issues commonly encountered in deep learning training when using FP16. This increased dynamic range makes it more robust for gradient calculations and weight updates. By providing a wider range while maintaining the same performance advantages of half-precision, BF16 has become the industry standard for stabilizing training and inference of modern large language models.

Answer

BF16 provides double the precision of FP16.

Answer

BF16 requires significantly less memory than FP16.

Answer

BF16 is faster on non-NVIDIA hardware.

Question 5

Refer to the exhibit. An engineer receives this timeout error during a CUDA kernel execution. What is the most appropriate first step to diagnose the resource contention?

Accepted Answer

Use NVIDIA Nsight Compute to profile occupancy.. CUDA kernel timeouts are typically caused by long-running kernels that exceed the GPU's watchdog timer or by excessive resource usage (registers/shared memory) that limits occupancy. Using the NVIDIA Nsight Compute profiler allows the engineer to see the exact occupancy metrics and register pressure, enabling targeted optimizations. This step is crucial for identifying if the kernel is over-provisioned for the specific GPU architecture being targeted.

Answer

Increase the block size to 2048 threads.

Answer

Disable the watchdog timer in the OS.

Answer

Switch the kernel to run on the CPU.

Question 6

Which hardware component of an NVIDIA GPU is most responsible for accelerating matrix-multiply-accumulate (MMA) operations used in transformer layers?

Accepted Answer

Tensor Cores.. Tensor Cores are specialized hardware units designed to perform high-speed matrix-multiply-accumulate operations in a single clock cycle. By accelerating these core operations, Tensor Cores provide the massive compute throughput needed for deep learning. Understanding the role of Tensor Cores is vital because they define the performance limits for modern LLMs, and optimizing code to utilize them is the single most important task in GPU performance tuning.

Answer

CUDA Cores.

Answer

Streaming Multiprocessor (SM) Scheduler.

Answer

L2 Cache controller.

Question 7

When profiling an application with NVIDIA Nsight Systems, which TWO metrics are most critical to identify if an application is limited by the PCIe bus?

Accepted Answer

Host-to-Device (H2D) throughput.. PCIe bus saturation occurs when the transfer rate of data between the host and the GPU becomes a bottleneck. By monitoring the H2D (Host-to-Device) and D2H (Device-to-Host) transfer metrics, developers can see if data movement consumes a disproportionate amount of execution time. Identifying these spikes is crucial because offloading data to the GPU is often the slowest part of a pipeline, and minimizing these transfers is key to scaling high-performance AI.

Answer

GPU Register usage count.

Answer

Shared memory bank conflict count.

Answer

SM clock frequency.

Question 8

Which of the following describes the purpose of 'Kernel Fusion' in the context of optimizing a Deep Learning inference pipeline?

Accepted Answer

To reduce redundant global memory read/write cycles.. Kernel fusion combines multiple small operations into a single GPU kernel to reduce the overhead of launching kernels and accessing global memory. Every kernel launch involves CPU-side overhead, and global memory accesses are costly in terms of energy and time. Fusion minimizes both, significantly increasing the effective throughput of the GPU by keeping data in high-speed, on-chip storage for as long as possible.

Answer

To increase the number of parallel GPU threads.

Answer

To enable multi-GPU distributed training.

Answer

To improve model accuracy through extra precision.

Question 9

Why is 'Pinned Memory' (page-locked) essential for high-performance data transfers between host and GPU?

Accepted Answer

It allows direct DMA access to host RAM.. Pinned memory prevents the operating system from swapping data to disk, allowing the GPU to access host memory directly via DMA (Direct Memory Access). This avoids the need for the driver to copy data into intermediate buffers, which is a major source of latency in standard data pipelines. By using pinned memory, applications can achieve significantly higher transfer speeds, which is vital for real-time generative AI applications.

Answer

It provides larger memory capacity on the GPU.

Answer

It enables automatic data compression.

Answer

It eliminates the need for CUDA contexts.

Question 10

When using the NVIDIA Collective Communications Library (NCCL), what does 'AllReduce' specifically optimize for in a multi-GPU training configuration?

Accepted Answer

Gradient aggregation across all devices.. AllReduce is critical for distributed training because it synchronizes the gradient updates from all GPUs across the cluster. It aggregates data from all devices and distributes the result back to each one, allowing models to train concurrently on massive datasets. By optimizing this collective operation, NCCL minimizes the time GPUs spend waiting for synchronization, which is the primary hurdle in scaling training to hundreds or thousands of GPUs.

Answer

Local GPU thread synchronization.

Answer

Direct CPU-to-GPU data copy.

Answer

Increasing GPU clock frequency.

Question 11

What is the primary function of the 'TensorRT' optimization engine in the NVIDIA AI software stack?

Accepted Answer

It performs architecture-specific model optimization.. TensorRT optimizes neural network models by performing layer fusion, precision calibration (e.g., to FP8 or INT8), and kernel selection optimized for the specific GPU architecture. By transforming the model into a highly efficient, platform-specific format, TensorRT significantly reduces latency and increases throughput for production inference. It is the core tool for moving from research-grade PyTorch models to production-ready deployments on NVIDIA hardware.

Answer

It automates the training of complex models.

Answer

It provides a Python API for model debugging.

Answer

It converts models to run on mobile CPUs.

Question 12

Which TWO of the following techniques are best suited for reducing the latency of LLM inference on NVIDIA GPUs?

Accepted Answer

KV cache quantization.. Reducing LLM latency requires optimizing both the compute throughput and the memory access patterns. Key-Value (KV) cache quantization and PagedAttention are industry standards for LLM acceleration. KV cache quantization reduces memory bandwidth usage, while PagedAttention manages memory dynamically, preventing fragmentation. These methods significantly improve the token generation rate, which is the primary metric for user-perceived performance in large-scale generative AI deployments.

Answer

Increasing the number of CPU worker threads.

Answer

Disabling ECC memory on the GPU.

Answer

Switching from Tensor Cores to CUDA Cores.

Question 13

When deploying a model using NVIDIA TensorRT, what is the primary benefit of the 'Engine Building' phase?

Accepted Answer

It performs target-specific kernel selection and layer fusion.. The TensorRT builder phase analyzes the model graph and hardware topology to select the most efficient kernels for the target GPU. This process includes layer fusion, precision calibration, and kernel selection, which are vital for production-grade inference. By tailoring the model specifically to the underlying hardware architecture, TensorRT achieves significantly higher throughput and lower latency than executing generic framework-native code directly on the GPU.

Answer

It converts the model to a generic portable format.

Answer

It automatically scales the model across multiple nodes.

Answer

It ensures the model can run on any CPU architecture.

Question 14

Refer to the exhibit. The model performance is inconsistent. What is the most likely reason for the performance variability under load?

Accepted Answer

Dynamic batching lacks a max_queue_delay_microseconds parameter.. The configuration uses dynamic batching with a preferred batch size range but lacks a 'max_queue_delay_microseconds' setting. Without a delay buffer, the server might dispatch batches as soon as one request arrives, leading to suboptimal under-filled batches and inconsistent latency. Adding a small delay allows the server to collect more requests, increasing throughput and stabilizing inference latency across fluctuating traffic patterns, which is vital for high-performance production deployments.

Answer

The instance count is too high for the GPU.

Answer

The GPU index is incorrectly defined in the config.

Answer

TensorRT plan files are inherently non-deterministic.

Question 15

When profiling an application with NVIDIA Nsight Systems, you notice a long gap between kernel execution blocks on the GPU timeline. What is the most likely cause?

Accepted Answer

Excessive host-side synchronization calls.. Long gaps in the GPU timeline usually indicate host-side synchronization, such as explicit cudaDeviceSynchronize() calls or CPU-bound code that is stalling the kernel launch queue. In production systems, unnecessary CPU-to-GPU synchronization forces the GPU to remain idle while waiting for the CPU to catch up, directly impacting system-wide latency and violating the principle of asynchronous task pipelining necessary for optimal GPU utilization.

Answer

The GPU is overheating and triggering thermal throttling.

Answer

The PCIe bus is saturated with high-frequency data.

Answer

The kernel is launching with an invalid thread block size.