NCP-GENL Fine-Tuning Practice Test 3 — 15 Questions

Question 1

Why is it important to use a 'warm-up' period in the learning rate schedule when starting a fine-tuning job?

Accepted Answer

To allow the optimizer to adapt to the new data distribution gradually.. A warm-up period gradually increases the learning rate from a near-zero value to the target rate. This prevents early, massive gradient updates from destabilizing the pre-trained weights. By slowly introducing the learning rate, the model remains stable during the initial phase of training, ensuring that the optimizer can effectively adapt to the new domain without destroying the fundamental knowledge captured during the original pre-training process.

Answer

To increase the total training time for better hardware utilization.

Answer

To force the model to explore a wider range of the loss landscape immediately.

Answer

To bypass the need for gradient clipping during training.

Question 2

When fine-tuning on a small, domain-specific dataset, why might adding synthetic data generated by a larger model be beneficial?

Accepted Answer

It helps the model generalize better by increasing data diversity.. Small datasets often lack the depth needed for a model to generalize effectively. Synthetic data can fill these gaps by providing more examples of the target task, which helps the model learn the nuances of the domain. This technique, when done correctly, reinforces desired behaviors and prevents overfitting on the limited original data, leading to a more robust, versatile, and high-performing model in the intended application area.

Answer

It eliminates the need for any real-world human-annotated data entirely.

Answer

It reduces the VRAM usage of the training process significantly.

Answer

It guarantees the model will not have any factual inaccuracies.

Question 3

A team is fine-tuning a Llama 2 7B model with NVIDIA NeMo Framework on a single A100 80GB GPU. They observe that training loss decreases initially but then diverges, and the model outputs become repetitive and incoherent. The team used a learning rate of 5e-5 with AdamW and no warm-up. Which change is most likely to stabilize training and improve convergence?

Accepted Answer

Reduce the learning rate to 1e-5 and add a linear warm-up over the first 10% of training steps.. The model diverges due to a learning rate that is too high for fine-tuning without warm-up. Reducing the learning rate to 1e-5 and adding a warm-up phase allows the optimizer to take smaller, more controlled steps initially, preventing large updates that disrupt pretrained weights. This combination is a well-established best practice in NVIDIA NeMo and other frameworks for stable fine-tuning of LLMs.

Answer

Increase the learning rate to 1e-4 to escape local minima.

Answer

Switch the optimizer to SGD with momentum 0.9 and keep the learning rate at 5e-5.

Answer

Increase the batch size to 64 and keep all other hyperparameters unchanged.

Question 4

A team is fine-tuning a 13B-parameter Llama model with NVIDIA NeMo on a node of eight A100 80GB GPUs. They want the optimizer state to be partitioned across data-parallel ranks so that per-GPU memory drops, while keeping the model replicas synchronized. Which distributed strategy should they select in the NeMo training configuration?

Accepted Answer

ZeRO Stage 1 via the distributed optimizer in Megatron Core. Sharding optimizer state across data-parallel ranks is exactly what the distributed optimizer in Megatron Core does, providing ZeRO Stage 1 behavior with full model replicas per rank. Tensor and pipeline parallelism change how parameters and layers are split, and recomputation targets activations. Only optimizer-state partitioning reduces the Adam moments and master weights each GPU must hold without adding model-parallel communication.

Answer

Tensor Parallelism (TP) with tensor_model_parallel_size set to 8

Answer

Pipeline Parallelism (PP) with pipeline_model_parallel_size set to 8

Answer

Activation recomputation with recompute_granularity set to full

Question 5

A team is fine-tuning a 13B-parameter Llama model with NVIDIA NeMo Framework on a single A100 80GB GPU. They apply LoRA adapters to the attention projection layers, but the adapters are producing negligible changes to model behavior even after several epochs, and the loss curve stays flat. They confirm the dataset is clean and the tokenizer is correct. Which LoRA configuration issue is the most likely cause?

Accepted Answer

The LoRA alpha scaling value is set extremely low relative to the rank, so the adapter's effective update magnitude is near zero.. LoRA updates are scaled by alpha divided by rank, so an alpha that is very small relative to rank shrinks the adapter's effective contribution and can leave the loss nearly unchanged. Correcting the alpha-to-rank ratio restores meaningful adapter influence, allowing the fine-tune to actually shift model behavior on the target task.

Answer

The optimizer was configured with a momentum value of zero, preventing the adapter weights from accumulating updates.

Answer

The LoRA target modules were set to the embedding layer rather than the attention projection layers.

Answer

The base model weights were accidentally left trainable, so gradients are flowing into the full model instead of the adapters.

Question 6

A team is fine-tuning an 8B-parameter LLM with LoRA on a single NVIDIA A100 80GB GPU using NVIDIA NeMo. They observe that training loss decreases, but validation loss starts to rise after epoch 2. They want to keep the same dataset and hyperparameters but mitigate overfitting. Which change is most appropriate?

Accepted Answer

Reduce the number of training epochs and apply early stopping based on validation loss.. The divergence between decreasing training loss and increasing validation loss after epoch 2 is a textbook overfitting signal. The most direct, minimal-risk remedy is to stop training earlier using validation loss as the criterion, preserving the checkpoint that generalizes best. Enlarging adapter capacity or altering optimizer and batch settings does not target the generalization gap and may worsen it.

Answer

Increase the batch size and scale the learning rate proportionally to speed convergence.

Answer

Switch the optimizer from AdamW to SGD with momentum to regularize the adapter.

Answer

Increase the LoRA rank from 8 to 64 to give the adapter more capacity.

Question 7

An ML engineer is fine-tuning a 70B model with NVIDIA NeMo Framework across 16 H100 GPUs. Training completes successfully, but when the fine-tuned checkpoint is evaluated, outputs are incoherent and repeat tokens. The engineer confirms the loss decreased smoothly during training and the validation dataset was held out correctly. Which issue is the most likely explanation?

Accepted Answer

The model was trained with a sequence packing strategy that concatenated unrelated samples without proper attention masking between them.. Sequence packing without correct cross-sample attention masking lets tokens attend to unrelated neighboring samples, teaching the model spurious dependencies that surface as repetition and incoherence at inference. The training loss can still fall smoothly because the model is fitting the corrupted attention structure, so loss alone does not reveal the defect.

Answer

The checkpoint was saved and later loaded with a mismatched tokenizer vocabulary, so input token IDs no longer map to the trained embeddings.

Answer

The evaluation prompts were longer than the maximum sequence length used during fine-tuning, so the model never learned to handle them.

Answer

The learning rate was too high, causing the optimizer to overshoot and permanently corrupt the base model weights.

Question 8

An engineer fine-tunes a model on a domain corpus with NVIDIA NeMo and observes that training loss falls steadily while validation loss begins rising after the second epoch. The team must produce the most generalizable checkpoint without changing the dataset. Which action should they take?

Accepted Answer

Enable early stopping based on validation loss and restore the best checkpoint. When training loss keeps falling but validation loss turns upward, the model is overfitting and the best generalizing weights occur near the point where validation loss is lowest. Early stopping with best-checkpoint restoration captures that point. More epochs, a higher learning rate, or dropping validation all fail to address the divergence or actively make it worse.

Answer

Increase the number of training epochs so the model fully converges on the domain corpus

Answer

Disable validation during training and rely on the final training loss to select the checkpoint

Answer

Raise the learning rate to escape the local minimum the model has settled into

Question 9

A developer needs to fine-tune a 7B LLM for a customer-support chatbot using NVIDIA NeMo. The dataset contains paired instructions and desired responses. Which data format should be used to prepare the dataset for supervised fine-tuning?

Accepted Answer

A JSONL file where each line contains an input text field and an output text field representing the instruction-response pair.. Supervised fine-tuning trains a model to produce a target response given an input instruction. The dataset must therefore contain aligned input-output pairs, and NeMo's data pipeline accepts JSONL records with distinct input and output fields. Files lacking response labels, pre-tokenized sequences without pairing, or pure configuration files cannot supply the supervision signal the training objective requires.

Answer

A YAML configuration file listing hyperparameters and dataset paths only.

Answer

A pickle file containing a Python list of tokenized integer sequences with no text labels.

Answer

A CSV file containing only the raw customer queries with no corresponding responses.

Question 10

A financial services company is fine-tuning an LLM to answer questions about internal policies. The base model performs well on general text but frequently invents policy numbers and effective dates. The team has a curated dataset of 5,000 question-answer pairs with correct citations. Which fine-tuning approach best addresses the hallucination of policy numbers and dates?

Accepted Answer

Perform supervised fine-tuning on the curated question-answer pairs so the model learns to produce answers with correct citations.. Supervised fine-tuning on curated question-answer pairs with correct citations is the most direct way to teach the model the desired behavior. The training examples pair each question with grounded policy numbers and effective dates, so the model learns to reproduce accurate citations instead of inventing them. Continued pre-training, length-based RLHF, and higher inference temperature do not target the specific hallucination of policy details.

Answer

Increase the model's temperature during inference so it explores a wider range of possible policy numbers and dates.

Answer

Apply reinforcement learning from human feedback using a reward model that penalizes any answer longer than two sentences.

Answer

Continue pre-training on a large corpus of internal documents to expand the model's knowledge of policy numbers and dates.

Question 11

A company wants to teach a pretrained LLM to follow a specific output format for customer support replies using supervised fine-tuning on NVIDIA GPUs. Which data preparation approach best matches supervised fine-tuning for instruction following?

Accepted Answer

Collect pairs of an instruction prompt and the desired response, then train the model to predict the response tokens. Instruction-following SFT learns from prompt and response pairs with the loss applied to response tokens, which directly teaches the desired output format. Continued pretraining on unlabeled text, RLHF with a reward model, and response-only training all lack the prompt-conditioned supervised signal that this task requires.

Answer

Score model outputs with a reward model and update the policy using a policy gradient objective

Answer

Provide only the desired responses without prompts and train the model to reproduce them verbatim

Answer

Gather a large unlabeled corpus and train the model to predict the next token across all documents

Question 12

A data science team is preparing an instruction fine-tuning dataset in NVIDIA NeMo Framework. They notice that after training, the model performs well on the training instructions but poorly on paraphrased versions of the same instructions. They want to improve generalization without increasing dataset size. Which data preparation change is most appropriate?

Accepted Answer

Apply instruction paraphrasing and template diversification so the same intent appears in multiple surface forms during training.. Generalization to paraphrased instructions depends on the model seeing the same intent expressed in varied surface forms. Paraphrasing and template diversification teach the model to respond to meaning rather than exact wording, directly improving performance on unseen paraphrases without enlarging the dataset or changing training hyperparameters.

Answer

Remove all examples where the instruction phrasing is unique, keeping only the most common templates.

Answer

Increase the number of epochs so the model sees each instruction more times and memorizes the exact phrasing.

Answer

Lower the learning rate so the model updates more slowly and avoids overfitting to specific instruction phrasings.

Question 13

An ML engineer is fine-tuning a 13B LLM with LoRA on 4 NVIDIA A100 GPUs using NVIDIA NeMo. They notice that the effective batch size is very small and gradients are noisy, but increasing the per-GPU micro batch size triggers out-of-memory errors. Which technique should they apply to increase the effective batch size without increasing memory per step?

Accepted Answer

Enable gradient accumulation to sum gradients across multiple micro batches before the optimizer step.. Gradient accumulation performs multiple forward and backward passes on small micro batches and sums their gradients before one optimizer update, emulating a larger batch while keeping peak memory at the level of a single micro batch. It is the standard remedy when memory prevents enlarging the micro batch. Changing adapter rank, precision, or sequence length does not increase the number of samples per update in a memory-neutral way.

Answer

Switch from bfloat16 to float32 precision to stabilize gradient values.

Answer

Increase the LoRA rank and alpha so the adapter captures more information per step.

Answer

Reduce the sequence length by truncating all training examples to 128 tokens.

Question 14

A developer is preparing a dataset for instruction fine-tuning of an LLM using NVIDIA NeMo. The raw data consists of customer support transcripts with speaker labels and timestamps. Which preprocessing step is most important before training?

Accepted Answer

Convert each transcript into a structured instruction-response pair with a clear prompt and a target completion.. Instruction fine-tuning requires examples that pair a prompt with the desired response so the model learns to follow instructions. Raw support transcripts contain speaker labels and timestamps that are not part of the target behavior. Converting each transcript into a structured instruction-response pair makes the data compatible with the training objective and ensures the loss is computed on the correct tokens.

Answer

Duplicate each transcript so the model sees every example at least twice per epoch.

Answer

Remove all punctuation and capitalization so the model learns a consistent lowercase style.

Answer

Increase the learning rate to compensate for the noisy transcript data.

Question 15

An enterprise is fine-tuning a 34B model with NVIDIA NeMo Framework and observes that the validation loss begins rising after the first epoch while training loss continues to fall. The team wants to reduce this divergence and preserve downstream task quality. (Choose two.)

Accepted Answer

Apply weight decay or increase its coefficient to penalize large parameter updates and discourage overfitting.. Rising validation loss with falling training loss is classic overfitting. Early stopping on validation loss halts training before further damage, while stronger weight decay regularizes the model and narrows the generalization gap. Together they reduce divergence and protect downstream task quality without discarding useful training signal.

Answer

Remove the validation split and train on all available data so the model sees more examples per epoch.

Answer

Raise the learning rate significantly so the model converges faster and reaches a flatter minimum.

Answer

Increase the number of training epochs so the optimizer has more time to escape the overfitting region.