NCP-GENL Practice Questions 1 — 50 Questions

Question 1

When profiling an application with NVIDIA Nsight Systems, which TWO metrics are most critical to identify if an application is limited by the PCIe bus?

Accepted Answer

Host-to-Device (H2D) throughput.. PCIe bus saturation occurs when the transfer rate of data between the host and the GPU becomes a bottleneck. By monitoring the H2D (Host-to-Device) and D2H (Device-to-Host) transfer metrics, developers can see if data movement consumes a disproportionate amount of execution time. Identifying these spikes is crucial because offloading data to the GPU is often the slowest part of a pipeline, and minimizing these transfers is key to scaling high-performance AI.

Answer

GPU Register usage count.

Answer

Shared memory bank conflict count.

Answer

SM clock frequency.

Question 2

Refer to the exhibit. Given this NeMo configuration, which prompt modification would best improve the reliability of technical support queries?

Accepted Answer

Require the model to state 'I cannot answer this' if the manual is silent.. The exhibit shows a relatively low temperature, which is good for consistency, but the system prompt lacks specific constraints on how to handle missing data. By modifying the prompt to strictly enforce grounding in the manuals and providing a fallback mechanism, you significantly improve the model's reliability in technical support scenarios. This is a critical step for maintaining quality in production environments utilizing NVIDIA NeMo technology.

Answer

Append 'Be as creative as possible' to the system prompt.

Answer

Change the stop sequences to be empty.

Answer

Increase the temperature to 0.9 to ensure varied answers.

Question 3

Refer to the exhibit. The Triton Inference Server is failing to load a model. The configuration shows two instances assigned to GPU 0. What is the most likely cause and the correct remediation?

Accepted Answer

The total VRAM required for two instances exceeds the GPU capacity; reduce instance count to 1.. The error indicates that the two instances of the model are collectively requesting more VRAM than is available on the physical GPU. By default, Triton attempts to allocate memory for every configured instance upon startup. The remediation requires either reducing the number of instances or implementing a memory-aware model partitioning strategy, such as using Model Analyzer to determine the safe memory footprint per instance before deployment.

Answer

The model count exceeds the number of available CUDA cores, requiring a driver update.

Answer

The model weights are corrupted, preventing the inference engine from initializing the memory space.

Answer

The batch size is set to zero in the configuration, preventing memory allocation.

Question 4

Which NVIDIA library is primarily used for optimizing and deploying deep learning inference models?

Accepted Answer

TensorRT.. TensorRT is NVIDIA's dedicated deep learning inference optimizer and runtime engine. It takes pre-trained models from frameworks like PyTorch or TensorFlow and optimizes them for NVIDIA hardware by fusing layers, selecting best-fit kernels, and performing precision calibration. It is the core tool for any developer needing to transition models from research training to production-level deployment with minimal latency and maximal throughput on NVIDIA infrastructure.

Answer

cuBLAS.

Answer

NCCL.

Answer

cuDNN.

Question 5

Which technique is most effective for mitigating data leakage during the training of an LLM on time-series-related document data?

Accepted Answer

Chronological splitting of the dataset based on a cutoff date.. Preventing data leakage in temporal datasets requires strict chronological separation between training, validation, and testing sets. If future information is inadvertently included in the training data, the model will 'cheat' by observing future events during training, leading to artificially inflated performance metrics that do not generalize to actual production deployment scenarios where future data is unavailable.

Answer

Random shuffling of all documents regardless of their timestamps.

Answer

Oversampling the minority class in the training set to improve balance.

Answer

Applying aggressive data normalization to all numerical values.

Question 6

Which hardware architecture feature is specifically leveraged by TensorRT to accelerate FP16 and INT8 matrix multiplications?

Accepted Answer

NVIDIA Tensor Cores. NVIDIA Tensor Cores are specialized hardware units designed to perform mixed-precision matrix multiply-accumulate operations in a single cycle. TensorRT automatically detects the presence of these cores and maps compute-intensive layers to them. By utilizing Tensor Cores, the model achieves significantly higher throughput and reduced latency compared to using standard CUDA cores, which perform operations at a lower efficiency per clock cycle for matrix math.

Answer

CPU AVX-512 vector instructions

Answer

Shared memory buffers in L1 cache

Answer

Global memory coalescing hardware

Question 7

Refer to the exhibit. An engineer is fine-tuning an LLM using the provided configuration. What is the primary purpose of applying 'yarn' scaling in this architecture?

Accepted Answer

It enables the model to process sequences longer than the original pre-trained limit.. YaRN (Yet another RoPE extension) is designed to extend the context window of pre-trained LLMs beyond their original training length. By modifying the Rotary Positional Embeddings (RoPE) base frequency, it allows the model to interpolate position indices without severe degradation in perplexity. This is essential for enterprise deployments requiring document retrieval or analysis tasks where input lengths exceed the base model's pre-trained constraints.

Answer

It increases the number of attention heads to improve feature extraction.

Answer

It compresses the model weights to reduce storage requirements.

Answer

It replaces the attention mechanism with a recurrent neural network.

Question 8

When optimizing a Generative AI model using NVIDIA TensorRT-LLM, which component is primarily responsible for managing the KV cache to minimize memory fragmentation?

Accepted Answer

The Paged KV Cache Manager. TensorRT-LLM utilizes PagedAttention or similar paged memory management techniques to handle the Key-Value (KV) cache. By dividing the cache into fixed-size blocks, the framework avoids the fragmentation issues inherent in contiguous memory allocation. This ensures that the memory is used efficiently during autoregressive generation, allowing for larger context windows and higher concurrent user capacity on the GPU.

Answer

The TensorRT Builder

Answer

The Model Parallelism Engine

Answer

The Triton Inference Server

Question 9

What is the primary function of the 'rank' parameter in LoRA?

Accepted Answer

It defines the dimension of the low-rank decomposition. The rank parameter ('r') determines the dimensionality of the update matrices injected into the model. A lower rank results in a smaller number of trainable parameters, which is more efficient but less expressive. A higher rank allows for more complex adaptations to the data. Balancing this rank is essential to achieve the desired model performance while staying within hardware memory limits.

Answer

It sets the total number of layers that are trainable

Answer

It dictates the number of epochs the model will train

Answer

It determines the learning rate for the adapter weights

Question 10

A team is training a 13B-parameter LLM on 8 NVIDIA A100 GPUs using NVIDIA NeMo. They observe that the all-reduce communication during data-parallel training consumes nearly 40% of each iteration. Which of the following changes is most likely to reduce this communication overhead while preserving convergence?

Accepted Answer

Use NVIDIA NCCL with the ring algorithm and overlap communication with computation via gradient bucketing.. The communication bottleneck in data-parallel training is best mitigated by using an efficient collective algorithm and hiding latency. NCCL's ring algorithm excels with large messages on high-bandwidth links, and gradient bucketing enables overlap with compute. This preserves the data-parallel semantics and convergence while reducing wall-clock time spent in all-reduce.

Answer

Switch from data parallelism to tensor parallelism across all 8 GPUs.

Answer

Enable gradient accumulation with a larger micro-batch size and use NCCL with tree algorithm.

Answer

Reduce the number of GPUs to 4 and increase the per-GPU batch size.

Question 11

When fine-tuning a Large Language Model using Low-Rank Adaptation (LoRA), which architectural component is primarily modified to reduce computational overhead while maintaining performance?

Accepted Answer

The attention weight projection matrices. LoRA freezes the pre-trained model weights and injects trainable rank decomposition matrices into the transformer architecture layers. By targeting specifically the attention query, key, and value projection matrices, practitioners can achieve high performance with a fraction of the trainable parameters. This approach is critical for memory-constrained environments, allowing fine-tuning on consumer-grade NVIDIA GPUs while avoiding the massive memory requirements associated with full parameter updates.

Answer

The entire embedding layer matrix

Answer

The feed-forward network activation functions

Answer

The entire decoder hidden state output

Question 12

When optimizing a model using NVIDIA TensorRT, what is the primary benefit of enabling 'layer fusion' during the optimization process?

Accepted Answer

It reduces global memory access overhead. Layer fusion reduces the overhead of launching multiple small GPU kernels by combining them into a single, optimized kernel. This strategy minimizes the total number of reads and writes to global VRAM, which is often the bottleneck in modern deep learning. By keeping intermediate data in high-speed registers or shared memory, the model spends less time waiting for memory access and more time performing actual floating-point operations.

Answer

It increases the overall model weight precision

Answer

It automatically prunes redundant parameters

Answer

It replaces floating-point math with integer math

Question 13

A team is deploying a 13B-parameter decoder-only LLM on a single NVIDIA A100 40GB GPU for a real-time chatbot. During load, the process runs out of memory even though the model weights in FP16 require roughly 26GB. The team wants to reduce GPU memory usage with minimal impact on output quality and no change to the model architecture. Which technique is most appropriate?

Accepted Answer

Apply INT8 weight-only quantization using NVIDIA TensorRT-LLM. The bottleneck is static weight storage on a single 40GB device. Weight-only INT8 quantization halves the parameter footprint while preserving the architecture and offering strong quality retention, and TensorRT-LLM provides a supported path for this on NVIDIA hardware. The other choices target attention compute, KV cache paging, or multi-GPU partitioning, none of which reduce the resident weight memory on one GPU.

Answer

Enable tensor parallelism across the single GPU's SMs

Answer

Switch the attention implementation to FlashAttention-2

Answer

Increase the KV cache block size to 32 tokens

Question 14

An LLM inference service on NVIDIA Triton Inference Server is experiencing occasional out-of-memory (OOM) errors on the GPU. The team wants to monitor GPU memory usage to predict and prevent OOM conditions. Which NVIDIA tool provides real-time GPU memory metrics that can be integrated with Prometheus for alerting?

Accepted Answer

NVIDIA Data Center GPU Manager (DCGM). DCGM is designed for data center GPU monitoring and provides comprehensive metrics, including memory usage. With DCGM Exporter, these metrics can be scraped by Prometheus, enabling real-time alerts when memory usage approaches limits. This proactive monitoring allows the team to adjust batch sizes or resource allocation before OOM errors occur, ensuring reliable LLM inference.

Answer

NVIDIA Nsight Systems

Answer

NVIDIA Triton Inference Server metrics endpoint

Answer

NVIDIA System Management Interface (nvidia-smi)

Question 15

What is the primary role of a Model Repository in the NVIDIA Triton Inference Server architecture?

Accepted Answer

To act as a centralized filesystem for serving multiple models.. The Model Repository acts as a centralized storage location for all deployed models, their configurations, and their versions. Triton monitors this directory to detect when new versions are added or updated, allowing for seamless model updates without restarting the server. This design supports robust MLOps practices by decoupling the model storage from the inference engine runtime, ensuring that deployments remain manageable and version-controlled.

Answer

To store training datasets for real-time model retraining.

Answer

To handle network traffic load balancing between server nodes.

Answer

To compile the model into an optimized executable format.

Question 16

Which TWO of the following techniques are best suited for reducing the latency of LLM inference on NVIDIA GPUs?

Accepted Answer

KV cache quantization.. Reducing LLM latency requires optimizing both the compute throughput and the memory access patterns. Key-Value (KV) cache quantization and PagedAttention are industry standards for LLM acceleration. KV cache quantization reduces memory bandwidth usage, while PagedAttention manages memory dynamically, preventing fragmentation. These methods significantly improve the token generation rate, which is the primary metric for user-perceived performance in large-scale generative AI deployments.

Answer

Increasing the number of CPU worker threads.

Answer

Disabling ECC memory on the GPU.

Answer

Switching from Tensor Cores to CUDA Cores.

Question 17

When preparing a proprietary technical manual dataset for a RAG pipeline, which data preprocessing step is most critical to ensure the LLM avoids hallucinations regarding specific product configurations?

Accepted Answer

Implementing document-aware recursive character splitting with overlapping segments and metadata tagging.. Chunking strategies and metadata tagging ensure that context retrieval is precise. By preserving technical hierarchy and associating data with specific product versions, the LLM retrieves ground-truth documentation rather than generic information. This reduces hallucinations by constraining the search space to relevant, version-controlled text blocks, directly impacting the accuracy and reliability of downstream inference tasks in enterprise NVIDIA-based AI deployments.

Answer

Converting all text to lowercase to ensure uniformity in vector embeddings.

Answer

Performing aggressive stop-word removal to reduce the dimensionality of the vector space.

Answer

Applying basic sentence tokenization based strictly on periods to create uniform chunks.

Question 18

What is the primary purpose of 'Few-Shot Prompting' in the context of LLM optimization?

Accepted Answer

To provide in-context learning examples to guide output.. Few-shot prompting involves providing a few examples of input-output pairs within the prompt to guide the model's performance on a specific task. This approach helps the model learn the desired format, tone, and logic without the need for intensive fine-tuning. It is a highly efficient way to steer model behavior for specific enterprise use cases, ensuring consistent results across multiple interaction sessions.

Answer

To reduce the latency of the underlying GPU cluster.

Answer

To compress the model weights for deployment.

Answer

To permanently store data in the model's internal memory.

Question 19

Which TWO actions should be part of a robust incident response plan for an LLM deployment failing in production?

Accepted Answer

Rolling back to a previous stable model version. A robust incident response plan focuses on rapid mitigation and root cause analysis. Immediately reverting to a known good version reduces the impact on users, while analyzing logs and telemetry provides the necessary data to understand the failure. These steps ensure that service is restored quickly, which is the primary objective of reliability engineering in generative AI systems serving critical user traffic.

Answer

Deploying the model to more GPUs immediately

Answer

Turning off all security monitoring systems

Answer

Forcing a reboot of all servers

Question 20

A research team wants to train a large decoder-only LLM where each token's representation is computed independently of token order, then inject order information afterward. They are choosing between learned absolute positional embeddings and sinusoidal absolute positional embeddings. Which statement accurately characterizes the tradeoff they face?

Accepted Answer

Sinusoidal embeddings are parameter-free and can be computed for any position, while learned embeddings require a fixed maximum length and add parameters.. Sinusoidal absolute positional embeddings are deterministic functions of position, adding no parameters and allowing evaluation at any index. Learned absolute embeddings allocate a trainable vector per position, consuming parameters and capping usable length at the trained maximum. The other options either reverse the extrapolation behavior or misattribute relative-offset abilities to learned absolute embeddings.

Answer

Learned embeddings are strictly better because they allow the model to discover relative offsets between tokens.

Answer

Learned embeddings can extrapolate to sequence lengths longer than those seen in training, whereas sinusoidal embeddings cannot.

Answer

Sinusoidal embeddings must be recomputed for every batch, which makes them slower than learned embeddings at inference.