NCP-GENL Fine-Tuning — 10 Questions

Question 1

Refer to the exhibit. Which adjustment should you make to the configuration if the model shows signs of overfitting on a small, domain-specific dataset?

Accepted Answer

Decrease lora_r to 8. Overfitting in LoRA often stems from excessive rank or an inappropriate alpha-to-rank ratio. Reducing 'lora_r' restricts the capacity of the adapter matrices, forcing the model to learn more generalized features. Conversely, adjusting 'lora_alpha' scales the impact of the adapters on the base model. Lowering these values acts as a form of regularization, preventing the model from memorizing the training data and improving its performance on unseen test samples.

Answer

Increase lora_r to 128

Answer

Change target_modules to include all linear layers

Answer

Increase lora_alpha to 64

Question 2

Refer to the exhibit. Which strategy is most effective to resolve the OOM error while maintaining the 70B parameter model's performance?

Accepted Answer

Implement QLoRA with 4-bit quantization and paged optimizers. A 70B model requires significant memory for weights and optimizer states, exceeding 24GB on a single GPU. Using QLoRA (4-bit quantization) with a paged optimizer allows the model to offload memory to system RAM and compress weights, making it possible to fit the model. This is the industry-standard approach for fine-tuning large models on limited consumer hardware without sacrificing too much precision or model capability.

Answer

Use a 64-bit optimizer instead of 32-bit

Answer

Increase the batch size to improve training stability

Answer

Switch to full fine-tuning without any weight freezing

Question 3

Which TWO metrics are most appropriate for evaluating the quality of a fine-tuned model for a creative writing task? (Choose two)

Accepted Answer

Human evaluation for coherence and style. Creative writing lacks a single ground-truth answer, making automated metrics like BLEU or ROUGE largely ineffective as they rely on exact n-gram matching. Instead, human evaluation ensures nuance, coherence, and stylistic adherence. Alternatively, using a more powerful 'LLM-as-a-judge' allows for consistent, scalable assessment of subjective qualities like creativity, which traditional metrics fail to capture in open-ended generative tasks.

Answer

Perplexity calculated on the test set

Answer

BLEU score against a reference text

Answer

Training loss on the fine-tuning dataset

Question 4

Which TWO of the following strategies are most effective for preventing catastrophic forgetting when fine-tuning a model for a new specialized domain?

Accepted Answer

Integrate a subset of the original pre-training data into the fine-tuning dataset.. Catastrophic forgetting occurs when new weights overwrite the knowledge captured during pre-training. Using a combination of a lower learning rate and mixing in a small percentage of general-purpose instruction data (rehearsal) ensures the model maintains its foundational logic while learning the new domain. These techniques are standard in NVIDIA-optimized workflows to maintain model versatility while achieving high performance in narrow vertical tasks.

Answer

Increase the learning rate significantly to force rapid adaptation to the new domain.

Answer

Freeze all layers except for the final output projection head.

Answer

Replace the entire attention mechanism with a standard feed-forward layer.

Question 5

When preparing a dataset for supervised fine-tuning (SFT), which THREE practices ensure high-quality model outcomes?

Accepted Answer

Ensuring the prompt-completion format is consistent across the entire dataset.. High-quality SFT depends on the diversity, format consistency, and accuracy of the training data. Ensuring the data reflects real-world use cases while maintaining clean, noise-free formatting prevents the model from learning incorrect patterns. These practices are foundational for any NVIDIA-based deployment, where data quality is often the primary driver of successful model performance compared to simply increasing the volume of training tokens.

Answer

Including as many low-quality, noisy samples as possible to increase volume.

Answer

Using an extremely large learning rate to force faster data ingestion.

Question 6

An enterprise AI engineer is preparing to fine-tune a large language model using NVIDIA NeMo. The training dataset contains highly confidential internal telemetry logs that must not be exposed to the underlying foundation model weights permanently or leaked during evaluation. Which specialized fine-tuning technique should be selected to minimize parameter modification while preserving core base model weights?

Accepted Answer

Low-Rank Adaptation (LoRA) keeping the pre-trained weights frozen and training low-rank rank decomposition matrices.. Parameter-Efficient Fine-Tuning methods like LoRA freeze original base model weights and inject trainable rank decomposition matrices. This prevents catastrophic forgetting and secures intellectual property by isolating updates into lightweight adapters, which is critical for enterprise data privacy requirements in production environments.

Answer

Full parameter fine-tuning across all transformer layers using distributed FP16 precision.

Answer

Prompt tuning by prepending trainable continuous embedding vectors to the frozen input sequences.

Answer

Quantization-aware training converting all transformer weights to INT4 precision prior to gradient updates.

Question 7

An enterprise AI team is preparing to fine-tune a large language model using NVIDIA NeMo. They want to ensure maximum training stability and avoid vanishing gradients when scaling up to multi-node clusters. Which activation checkpointing strategy should be configured?

Accepted Answer

Configure selective activation checkpointing to recompute specific bottleneck layers while retaining key tensor states in memory.. Selective activation checkpointing stores only critical activations during the forward pass and recomputes the rest during backward propagation. This reduces GPU memory overhead significantly while maintaining numerical stability, allowing larger batch sizes and preventing out-of-memory errors on NVIDIA hardware during large-scale model training.

Answer

Disable activation checkpointing entirely to maximize raw forward-pass throughput across the distributed cluster nodes.

Answer

Apply full activation checkpointing to every transformer layer indiscriminately to achieve the absolute lowest memory footprint possible.

Answer

Rely solely on CPU offloading for all intermediate activations to bypass GPU memory limits without altering checkpointing policies.

Question 8

When configuring Parameter-Efficient Fine-Tuning (PEFT) using LoRA within the NVIDIA NeMo framework, which TWO parameters are critical to adjust for balancing adaptation capacity and GPU memory consumption?

Accepted Answer

lora_rank (r). The rank (r) determines the dimensionality of the update matrices, directly controlling parameter capacity and memory usage. Scaling alpha (alpha) acts as a constant multiplier for the LoRA activations, ensuring stable learning dynamics when varying the rank. Both parameters must be tuned carefully together on NVIDIA hardware.

Answer

transformer_engine

Answer

tensor_parallel_size

Answer

global_batch_size

Question 9

An engineer is fine-tuning a 13B parameter model using NVIDIA NeMo Framework with tensor parallelism (TP) of 4 and pipeline parallelism (PP) of 2 on 8 A100 GPUs. They want to maximize throughput while maintaining convergence. They notice that increasing the micro-batch size beyond 4 causes an out-of-memory error, but GPU utilization remains low. Which configuration change should they make to improve throughput without increasing memory usage?

Accepted Answer

Increase the number of gradient accumulation steps and keep the micro-batch size at 4.. Gradient accumulation is the optimal solution because it increases the effective batch size without requiring additional GPU memory. By accumulating gradients over several micro-batches, the model can process more data per update, improving throughput and convergence. This approach leverages existing hardware efficiently and is a standard technique in NVIDIA NeMo for memory-constrained fine-tuning.

Answer

Increase the tensor parallelism degree to 8 and reduce pipeline parallelism to 1.

Answer

Enable activation recomputation (gradient checkpointing) and increase micro-batch size to 8.

Answer

Reduce the micro-batch size to 2 and double the number of GPUs to 16.

Question 10

A team is fine-tuning a 13B-parameter LLM with NVIDIA NeMo on a node of eight A100 80GB GPUs. They want to maximize training throughput while keeping the global batch size fixed and avoiding GPU memory errors. Which NeMo configuration change is most appropriate?

Accepted Answer

Keep data parallelism with Distributed Data Parallel, use gradient accumulation to reach the global batch size, and tune the micro-batch size to the largest value that fits in memory.. With a 13B model on eight 80GB GPUs, data parallelism plus Distributed Data Parallel is the standard efficient approach. Gradient accumulation preserves the desired global batch size when the micro-batch must remain modest, and tuning the micro-batch to the largest value that fits in memory maximizes GPU utilization. Model-parallel techniques add overhead that is unnecessary when the model already fits per GPU.

Answer

Switch to fully sharded data parallelism and reduce the global batch size to match the number of GPUs.

Answer

Enable activation recomputation for every transformer layer and increase the number of pipeline stages to eight.

Answer

Enable tensor parallelism and pipeline parallelism within the node, then increase micro-batch size until memory is nearly full.