NCP-GENL Practice Test 24 — 15 Questions

Question 1

A team is training a large language model on 8 NVIDIA A100 GPUs using PyTorch's DistributedDataParallel (DDP). Profiling shows that all GPUs are frequently idle, waiting for gradient synchronization. The network interconnect between nodes is a 100 Gb Ethernet with TCP/IP, and the model has 13 billion parameters. What is the most effective optimization to reduce the idle time?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Replace the Ethernet interconnect with NVIDIA Mellanox InfiniBand and enable NCCL over RDMA.. The idle time is caused by slow gradient synchronization over TCP/IP Ethernet. InfiniBand with RDMA and NCCL provides the necessary bandwidth and low latency to reduce all-reduce time, directly addressing the bottleneck. Other options either do not target communication or are secondary optimizations that do not resolve the fundamental interconnect limitation.

Answer

Increase the batch size per GPU to keep the GPUs busy during communication.

Answer

Enable gradient compression using FP16 all-reduce to halve communication volume.

Answer

Use NVIDIA GPUDirect Storage to accelerate data loading from NVMe drives.

Question 2

When auditing an NVIDIA Triton deployment for security, which action is most critical to protect sensitive inference data?

Accepted Answer

Enabling TLS for client-server communication. Securing the communication channel is the first line of defense in protecting sensitive user input data sent to an LLM. Utilizing TLS/SSL encryption prevents man-in-the-middle attacks where data could be intercepted during transit. In professional production environments, this is a standard requirement for compliance and data privacy, ensuring that prompt data and generated completions remain confidential between the client and the inference server.

Answer

Disabling the GPU logging

Answer

Increasing the memory buffer

Answer

Switching to an unsecured port

Question 3

A multinational insurer deploys an NVIDIA NIM-based claims triage assistant across the EU and Brazil. The compliance team must demonstrate that the system honors data-subject deletion requests and that personal data is not transferred outside approved regions. Which design decision addresses both obligations most directly?

Accepted Answer

Deploy regional NIM endpoints and per-region data stores so personal data stays in its jurisdiction, and implement deletion by cascading erasure across the prompt logs, vector index, and cached responses for that subject.. Residency is a placement problem and erasure is a propagation problem, so the design must solve both: region-local NIM endpoints and stores for placement, and cascading deletion across logs, indexes, and caches for propagation. Centralized logging, global replication of encrypted data, and edge anonymization each fail at least one obligation, and none of them produces a defensible deletion trail. Regional architecture with explicit erasure workflows is what an auditor can verify.

Answer

Anonymize prompts before inference and retain only aggregate statistics, discarding the original records immediately.

Answer

Encrypt all personal data with a single key managed by the home-region security team and replicate the encrypted stores globally for resilience.

Answer

Store all prompts and completions in a single centralized log bucket in the insurer's home region for simplified auditing.

Question 4

A retail company wants to release an LLM-powered shopping assistant on NVIDIA NIM. Legal requires that the assistant never provide personalized financial advice, even if a user asks for it. Which control most directly enforces this boundary at runtime?

Accepted Answer

Add a NeMo Guardrails dialog rail that detects financial-advice intent and returns a fixed refusal message before the LLM is invoked.. Deterministic runtime boundaries are established by guardrails that intercept intent before generation. A dialog rail in NeMo Guardrails can recognize financial-advice requests and return a fixed refusal, ensuring the model never produces the prohibited content. System prompts and token limits are probabilistic or unrelated controls, and policy documents do not enforce behavior.

Answer

Include a sentence in the system prompt instructing the model to avoid financial advice.

Answer

Reduce the model's maximum output tokens so responses are too short to contain financial advice.

Answer

Publish a terms-of-service page stating that the assistant does not provide financial advice.

Question 5

An ML engineer is fine-tuning a 70B model with NVIDIA NeMo Framework across 16 H100 GPUs. Training completes successfully, but when the fine-tuned checkpoint is evaluated, outputs are incoherent and repeat tokens. The engineer confirms the loss decreased smoothly during training and the validation dataset was held out correctly. Which issue is the most likely explanation?

Accepted Answer

The model was trained with a sequence packing strategy that concatenated unrelated samples without proper attention masking between them.. Sequence packing without correct cross-sample attention masking lets tokens attend to unrelated neighboring samples, teaching the model spurious dependencies that surface as repetition and incoherence at inference. The training loss can still fall smoothly because the model is fitting the corrupted attention structure, so loss alone does not reveal the defect.

Answer

The checkpoint was saved and later loaded with a mismatched tokenizer vocabulary, so input token IDs no longer map to the trained embeddings.

Answer

The evaluation prompts were longer than the maximum sequence length used during fine-tuning, so the model never learned to handle them.

Answer

The learning rate was too high, causing the optimizer to overshoot and permanently corrupt the base model weights.

Question 6

A production LLM inference service on NVIDIA Triton Inference Server is being monitored for reliability. The team wants to implement effective logging to diagnose issues such as high latency and errors. Which TWO logging practices are recommended for a production LLM environment? (Choose two.)

Accepted Answer

Use structured logging (e.g., JSON) to enable efficient parsing and querying.. Including a unique request ID enables end-to-end tracing, which is vital for diagnosing latency and errors in distributed LLM inference pipelines. Structured logging facilitates automated parsing and querying, allowing efficient monitoring and alerting. Together, they provide robust observability without excessive overhead or security risks.

Answer

Log every inference request and response payload for full traceability.

Answer

Disable logging in production to maximize inference throughput.

Answer

Log GPU temperature and power metrics at debug level for every inference.

Question 7

A team is deploying an NVIDIA NIM for a Llama 3 model as a retrieval-augmented generation (RAG) assistant over internal documentation. Users report that the assistant sometimes answers from its pretrained knowledge instead of the retrieved passages, and occasionally cites a passage that does not support its claim. Which TWO prompt engineering changes best reduce these behaviors? (Choose two.)

Accepted Answer

Instruct the model to answer only from the provided context and to respond with a fixed phrase when the context is insufficient.. Grounding the model with an explicit instruction to use only the provided context, plus a fallback for insufficient information, prevents reliance on pretrained knowledge. Requiring verbatim supporting quotes makes each claim auditable and discourages fabricated citations. Together these changes enforce source adherence and citation accuracy, while sampling changes or removing context would undermine the RAG design.

Answer

Add a chain-of-thought instruction asking the model to reason about why the user's question is interesting before answering.

Answer

Raise the temperature to 0.8 so the model produces more varied answers and avoids memorized responses.

Answer

Remove the retrieved passages from the prompt and rely on the model's pretrained knowledge to keep the prompt short.

Question 8

An enterprise is deploying a large language model on NVIDIA Triton Inference Server. Which deployment strategy minimizes latency for requests that require high-throughput batching while maintaining consistent hardware utilization?

Accepted Answer

Enable Dynamic Batching in the Triton model configuration file.. Dynamic Batching is the optimal strategy for Triton Inference Server in production environments. It groups individual inference requests arriving within a short time window into a single batch, allowing the GPU to process them in parallel. This maximizes throughput by fully saturating CUDA cores, reducing the overhead of kernel launches, and ensuring that hardware utilization remains high even under variable traffic loads, effectively balancing latency and overall system capacity.

Answer

Implement static batching with a fixed size of 1.

Answer

Disable all batching features to process requests serially.

Answer

Offload all batching logic to the client-side application layer.

Question 9

You are preparing a customer-support dataset for fine-tuning an LLM with NVIDIA NeMo. The raw data includes personally identifiable information such as names, email addresses, and phone numbers. Which data preparation step must be performed before training to comply with privacy requirements?

Accepted Answer

Apply PII detection and redaction to replace sensitive entities with placeholders.. Privacy compliance requires removing or masking personally identifiable information before training. PII detection and redaction replaces sensitive entities with placeholders, preserving the dataset's instructional value while preventing the model from memorizing real customer details. Tokenization, batch size, and shuffling do not alter the presence of PII and therefore cannot satisfy the requirement to protect sensitive data during fine-tuning.

Answer

Shuffle the dataset to break associations between PII and responses.

Answer

Increase the batch size during fine-tuning to average out PII exposure.

Answer

Tokenize the dataset with a custom vocabulary that includes PII patterns.

Question 10

A company wants to teach a pretrained LLM to follow a specific output format for customer support replies using supervised fine-tuning on NVIDIA GPUs. Which data preparation approach best matches supervised fine-tuning for instruction following?

Accepted Answer

Collect pairs of an instruction prompt and the desired response, then train the model to predict the response tokens. Instruction-following SFT learns from prompt and response pairs with the loss applied to response tokens, which directly teaches the desired output format. Continued pretraining on unlabeled text, RLHF with a reward model, and response-only training all lack the prompt-conditioned supervised signal that this task requires.

Answer

Score model outputs with a reward model and update the policy using a policy gradient objective

Answer

Provide only the desired responses without prompts and train the model to reproduce them verbatim

Answer

Gather a large unlabeled corpus and train the model to predict the next token across all documents

Question 11

Which metric provides the best indication of 'inference queue saturation' in a Triton deployment?

Accepted Answer

Request queue duration. Queue duration is the most direct indicator of saturation. It measures how long an inference request spends waiting in the server's input buffer before it is processed by the GPU. In a production environment, monitoring this metric is crucial to identify when the server's request capacity has been exceeded, allowing for auto-scaling triggers to provision more instances and maintain performance standards during traffic spikes.

Answer

Total GPU memory usage

Answer

GPU temperature

Answer

System clock speed

Question 12

A data engineer is preparing a JSONL instruction dataset for an NVIDIA NeMo supervised fine-tuning run. Each line currently contains a free-form 'text' field with the instruction, context, and response concatenated. The training configuration expects the standard NeMo instruction-tuning schema with separate fields for the task instruction, optional context, and the expected response. What is the most appropriate data preparation step?

Accepted Answer

Parse each record and rewrite it into the expected fields, such as instruction, input, and output, while preserving the original text content and escaping any embedded quotes.. NeMo's supervised fine-tuning loader expects separate instruction, context, and response fields, so the free-form text must be parsed into that schema with content preserved and quotes escaped. This produces a dataset the standard training configuration can consume directly, keeping the run reproducible and allowing loss to be computed on the response portion alone.

Answer

Keep the free-form text field and modify the NeMo training configuration to treat the entire line as the response.

Answer

Convert the JSONL file to plain text with one example per paragraph and let the tokenizer infer the instruction and response boundaries.

Answer

Duplicate each record and label one copy as instruction and the other as response so the loader sees two fields per example.

Question 13

You are evaluating a fine-tuned Llama-3-70B model for a customer service chatbot using NVIDIA NeMo Evaluation. The model's outputs are factually correct but often verbose, exceeding the desired response length. Which metric should you prioritize to quantify this issue?

Accepted Answer

Average response length. The core issue is that responses are too long, so the evaluation must quantify length. Average response length provides a direct, interpretable measure of verbosity. Other metrics like ROUGE-L, BLEU, and perplexity assess content overlap or fluency, not length, and thus cannot reliably diagnose the problem.

Answer

BLEU

Answer

Perplexity

Answer

ROUGE-L

Question 14

Which optimization technique specifically helps to manage the memory bandwidth bottleneck during the autoregressive decoding phase of an LLM?

Accepted Answer

KV Cache Quantization. During decoding, the model must read all KV cache values for every generated token. This is memory-bandwidth bound. Techniques like Quantized KV Cache (reducing the precision of cached tokens) and PagedAttention significantly reduce the amount of data moved between the VRAM and the compute units, effectively alleviating the memory bottleneck and allowing for faster token generation rates.

Answer

Operator Fusion

Answer

Weight Pruning

Answer

Dynamic Batching

Question 15

What is the primary benefit of deploying a model with a 'Model Ensemble' configuration in Triton Inference Server?

Accepted Answer

It enables multi-model pipelining with reduced network overhead.. Model ensembles allow developers to chain multiple models together into a single pipeline (e.g., preprocessing, inference, and post-processing). This keeps the logic within the server, reducing the overhead of multiple network round-trips between the client and the server. By managing the entire pipeline as a single unit, you ensure consistent data flow and lower overall latency for the end-to-end application.

Answer

It increases the number of GPUs available to the model.

Answer

It automatically converts models into TensorRT engines.

Answer

It provides a load-balancing mechanism across server clusters.