NCP-GENL Practice Test 8 — 10 Questions

Question 1

After fine-tuning a code-generation model with NVIDIA NeMo, an engineer notices the model now produces correct domain-specific function calls but has started emitting malformed JSON in about 15 percent of structured-output requests. The fine-tuning dataset contained no structured-output examples. Which evaluation action best explains and catches this regression?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Add a format-compliance check plus a general-capability regression suite to the evaluation, and compare against the base model on the same structured-output prompts.. Fine-tuning on a narrow dataset can degrade capabilities absent from that data, and structured output is a classic casualty. The right response is to measure it: add format-compliance scoring to the harness, rerun the same structured prompts against the base and tuned models, and include a general regression suite. Retraining harder, raising temperature, or dismissing the failures all skip the measurement step the situation demands.

Answer

Increase the temperature of the structured-output requests so the model has more freedom to produce valid JSON.

Answer

Conclude that the fine-tune succeeded on its target task and that the JSON failures are acceptable collateral given the domain gains.

Answer

Retrain the model with a larger learning rate so the structured-output behavior is reinforced more strongly during fine-tuning.

Question 2

A team is using an NVIDIA NIM-hosted Llama model to generate product descriptions from a list of technical specifications. The descriptions sometimes omit key specifications or include invented features. The team wants to improve reliability without changing the model. Which prompt engineering change is most likely to reduce these errors?

Accepted Answer

Add a system prompt that instructs the model to act as a technical writer and to include every specification exactly as given, and provide a few-shot example of a correct description.. To reduce omissions and inventions without changing the model, the most effective prompt engineering approach is to provide explicit instructions and few-shot examples. A system prompt that defines the role and constraints, combined with an example that demonstrates including all specifications and avoiding fabrication, guides the model to produce more reliable outputs. Other parameter changes or chain-of-thought do not directly address content fidelity.

Answer

Increase the max_tokens parameter to allow the model to generate longer descriptions, ensuring all specifications are covered.

Answer

Use a chain-of-thought prompt that asks the model to list each specification and then write the description.

Answer

Set the temperature to 0.0 to make the model deterministic and reduce creativity.

Question 3

You are preparing a large instruction-tuning dataset for an NVIDIA NeMo-based LLM. The raw data consists of user queries and assistant responses collected from a customer support system, stored as JSON lines. During preprocessing, you notice that many responses contain personally identifiable information (PII) such as names, email addresses, and phone numbers. You need to ensure the dataset is safe for training while preserving as much semantic content as possible. Which approach is most appropriate for handling PII in this dataset?

Accepted Answer

Use a named entity recognition (NER) model to detect PII and replace each entity with a generic placeholder (e.g., [NAME]).. Replacing PII with generic placeholders using an NER model is the most balanced approach: it removes sensitive information while preserving the linguistic patterns and context needed for instruction tuning. This method aligns with NVIDIA NeMo's data preprocessing recommendations for responsible AI. It also avoids the pitfalls of discarding data or introducing unintelligible tokens.

Answer

Leave the PII intact but exclude the dataset from production use and train only in a sandbox environment.

Answer

Hash each detected PII string with SHA-256 and substitute the hash into the text.

Answer

Apply a rule-based regex filter to remove any line containing PII, then train on the remaining data.

Question 4

Refer to the exhibit. An engineer receives this timeout error during a CUDA kernel execution. What is the most appropriate first step to diagnose the resource contention?

Accepted Answer

Use NVIDIA Nsight Compute to profile occupancy.. CUDA kernel timeouts are typically caused by long-running kernels that exceed the GPU's watchdog timer or by excessive resource usage (registers/shared memory) that limits occupancy. Using the NVIDIA Nsight Compute profiler allows the engineer to see the exact occupancy metrics and register pressure, enabling targeted optimizations. This step is crucial for identifying if the kernel is over-provisioned for the specific GPU architecture being targeted.

Answer

Increase the block size to 2048 threads.

Answer

Disable the watchdog timer in the OS.

Answer

Switch the kernel to run on the CPU.

Question 5

Refer to the exhibit. The deployment is facing memory allocation errors during peak load. Based on the error log, what is the most effective configuration change to resolve the issue while keeping the model architecture constant?

Accepted Answer

Enable PagedAttention in the TensorRT-LLM runtime.. The error log indicates that the system is running out of memory because the KV cache is allocated statically, which often leads to fragmentation or over-allocation. Enabling PagedAttention is the standard NVIDIA-recommended solution for this scenario. It allows the system to manage KV cache memory dynamically, reclaiming space from finished requests and efficiently allocating blocks for new tokens, thereby preventing the out-of-memory (OOM) errors caused by static allocation.

Answer

Switch to a smaller model size.

Answer

Increase the GPU clock frequency.

Answer

Reduce the batch size to one.

Question 6

Refer to the exhibit. An engineer observes that GPU memory utilization is high, but the GPU is frequently idling. How does the provided Triton configuration optimize the inference pipeline?

Accepted Answer

It enables concurrent execution on the GPU. By setting the instance count to 2 on a single GPU, the configuration enables concurrent model execution. This allows Triton to schedule multiple inference requests to be processed simultaneously on the same hardware. This overlap helps to hide memory latency and pipeline stalls, effectively increasing the utilization of the GPU and raising the overall throughput of the deployment.

Answer

It forces the model to use INT8 precision

Answer

It partitions the GPU for dedicated memory

Answer

It enables dynamic batching for the requests

Question 7

When using the NVIDIA Collective Communications Library (NCCL), what does 'AllReduce' specifically optimize for in a multi-GPU training configuration?

Accepted Answer

Gradient aggregation across all devices.. AllReduce is critical for distributed training because it synchronizes the gradient updates from all GPUs across the cluster. It aggregates data from all devices and distributes the result back to each one, allowing models to train concurrently on massive datasets. By optimizing this collective operation, NCCL minimizes the time GPUs spend waiting for synchronization, which is the primary hurdle in scaling training to hundreds or thousands of GPUs.

Answer

Local GPU thread synchronization.

Answer

Direct CPU-to-GPU data copy.

Answer

Increasing GPU clock frequency.

Question 8

Refer to the exhibit. What is the effective batch size for this fine-tuning job?

Accepted Answer

16. The effective batch size is calculated by multiplying the per-device batch size by the gradient accumulation steps. In this case, 1 * 16 equals 16. Gradient accumulation allows the model to simulate a larger batch size by running multiple forward and backward passes before performing a single optimizer step, which helps in achieving more stable gradient updates without increasing the immediate memory overhead for activations.

Answer

1

Answer

15

Answer

17

Question 9

Which metric is the most reliable indicator that an LLM is overfitting during the fine-tuning phase?

Accepted Answer

A decrease in training loss with a simultaneous increase in validation loss.. An increasing validation loss alongside a decreasing training loss is the classic indicator of overfitting. The model is essentially memorizing the training samples rather than learning generalized language patterns. Monitoring this divergence is critical in NVIDIA-based training pipelines, as it allows researchers to implement early stopping or adjust regularization strategies before the model's performance on unseen tasks degrades significantly.

Answer

A consistent increase in both training and validation loss.

Answer

A plateau in the training loss while the validation loss remains stagnant.

Answer

The model generates text that matches the training set exactly.

Question 10

You are preparing a massive dataset for training a NeMo-based LLM. Which TWO data preprocessing steps are critical to prevent data leakage and ensure model quality?

Accepted Answer

Performing temporal or content-based splitting to isolate the validation set.. Data leakage occurs when test data is inadvertently included in the training set, leading to inflated performance metrics. Deduplication is equally vital, as repetitive data causes the model to memorize samples rather than generalize. In NVIDIA workflows, these steps are typically performed via distributed scripts on the cluster before tokenization, ensuring that the model learns unique, non-overlapping information across all shards.

Answer

Increasing the frequency of data augmentation in the training pipeline.

Answer

Adding metadata tags to all training sequences for classification.

Answer

Converting all text to lowercase to increase vocabulary efficiency.