Fine-Tuning practice questions

Increase lora_r to 128

Why it fails: Increasing the rank parameter expands the capacity of the adapter matrices, which likely exacerbates overfitting. When a model overfits, it has already captured too much noise from the training set; providing more trainable parameters allows the model to memorize further, worsening the generalization gap rather than solving the issue.

Change target_modules to include all linear layers

Why it fails: Expanding the target modules increases the total count of trainable parameters, which typically increases the likelihood of overfitting on small datasets. For high-quality fine-tuning on limited data, fewer targeted modules are usually preferred to maintain regularization and focus the training on the most impactful attention mechanisms within the model.

Increase lora_alpha to 64

Why it fails: Increasing the alpha parameter effectively increases the weight of the LoRA adapters relative to the pre-trained weights. This makes the model more sensitive to the fine-tuning data, which would likely worsen overfitting rather than improving generalization. Alpha should generally be reduced or kept proportional to rank for stability.

Use a 64-bit optimizer instead of 32-bit

Why it fails: Moving to a 64-bit optimizer would double the memory usage of the optimizer states, making the Out of Memory (OOM) error significantly worse. Optimizer states are already a major contributor to memory consumption, and increasing their precision is the opposite of what is needed to resolve memory bottlenecks.

Increase the batch size to improve training stability

Why it fails: Increasing the batch size consumes more VRAM because it requires storing additional activation maps for every sample in the batch. If the model is already hitting OOM, increasing the batch size will only make the error occur sooner during the forward pass of the training loop.

Switch to full fine-tuning without any weight freezing

Why it fails: Full fine-tuning requires storing gradients and optimizer states for all 70 billion parameters, which would require hundreds of gigabytes of VRAM. This is physically impossible on a 24GB GPU and would immediately trigger an OOM error, making it the least viable strategy for this specific hardware constraint.

Perplexity calculated on the test set

Why it fails: Perplexity measures how well the model predicts the next token, which is a measure of statistical likelihood, not creative quality. A model might have low perplexity but produce repetitive or boring content. Therefore, it is a poor metric for evaluating whether a model is effectively 'creative' in its writing.

BLEU score against a reference text

Why it fails: BLEU measures n-gram overlap, which is useful for translation but detrimental for creative writing. A creative model should not produce an output that is a near-copy of a reference text. Relying on BLEU will penalize the model for being original, which is the opposite of the intended goal.

Training loss on the fine-tuning dataset

Why it fails: Training loss only indicates how well the model is memorizing the specific samples in the training set. It is not an evaluation metric for the model's performance on unseen creative prompts. A model can have very low training loss while completely failing to generalize or produce high-quality creative text.

Increase the learning rate significantly to force rapid adaptation to the new domain.

Why it fails: A high learning rate frequently accelerates the destruction of previously learned information. During fine-tuning, aggressive updates cause the weights to diverge from the pre-trained manifold, which leads to a loss of general reasoning capabilities and causes the model to overfit rapidly to the new, narrow dataset.

Freeze all layers except for the final output projection head.

Why it fails: Freezing all layers except the output head is usually insufficient for deep domain adaptation in LLMs. The model needs to adjust internal representations in the deeper transformer layers to grasp complex domain nuances, and simple output-layer fine-tuning fails to capture the necessary semantic shifts required for high-quality generation.

Replace the entire attention mechanism with a standard feed-forward layer.

Why it fails: Replacing core components like the attention mechanism disrupts the fundamental architecture of the transformer, rendering the pre-trained weights useless. Fine-tuning should preserve the architectural integrity of the model to ensure it remains capable of processing complex tokens and dependencies, rather than breaking the core model structure.

Including as many low-quality, noisy samples as possible to increase volume.

Why it fails: Low-quality data introduces noise and incorrect patterns, which the model will learn as ground truth. This leads to degraded performance in real-world applications. It is significantly more effective to curate a smaller, high-quality dataset than to rely on massive, noisy inputs that pollute the training process.

Using an extremely large learning rate to force faster data ingestion.

Why it fails: An extremely large learning rate causes unstable gradients and rapid loss of generalization, regardless of dataset quality. Proper training requires a measured, tuned learning rate that allows the model to adjust incrementally to the provided data, ensuring the learning process is stable and produces reliable results.

Full parameter fine-tuning across all transformer layers using distributed FP16 precision.

Why it fails: Full fine-tuning updates every single parameter in the transformer network, requiring massive GPU memory and risking catastrophic forgetting of foundational capabilities while failing to isolate proprietary telemetry data effectively. Heightened memory overhead makes this approach sub-optimal for constrained corporate setups.

Prompt tuning by prepending trainable continuous embedding vectors to the frozen input sequences.

Why it fails: Prompt tuning modifies only soft prompt embeddings appended to inputs, which limits adaptation capacity for complex domain-specific tasks like parsing technical telemetry logs compared to weight-modifying methods like LoRA. It struggles with deep structural reasoning adjustments.

Quantization-aware training converting all transformer weights to INT4 precision prior to gradient updates.

Why it fails: Quantization-aware training alters every transformer weight to INT4, permanently modifying the base model and degrading evaluation fidelity, which the requirement forbids. It tempts for reducing memory and inference cost, but that scenario is deployment optimisation, not the parameter-efficient fine-tuning needed here.

Disable activation checkpointing entirely to maximize raw forward-pass throughput across the distributed cluster nodes.

Why it fails: Disabling activation checkpointing causes massive memory consumption because every intermediate activation must be retained in VRAM. This inevitably leads to out-of-memory errors when training large-scale models, even on high-end NVIDIA enterprise GPUs with substantial memory capacity.

Apply full activation checkpointing to every transformer layer indiscriminately to achieve the absolute lowest memory footprint possible.

Why it fails: Applying full activation checkpointing to every single layer drastically increases recomputation overhead during the backward pass. This severely degrades training throughput across the cluster, making it an inefficient choice for standard enterprise fine-tuning pipelines.

Rely solely on CPU offloading for all intermediate activations to bypass GPU memory limits without altering checkpointing policies.

Why it fails: Relying solely on CPU offloading for intermediate activations creates severe PCIe bandwidth bottlenecks between the host CPU and the NVIDIA GPUs. This stalls the training loop and destroys distributed training scaling efficiency.

transformer_engine

Why it fails: Transformer Engine is an overarching library component designed to optimize FP8 and mixed-precision execution on NVIDIA Hopper GPUs. It is not a configurable PEFT hyperparameter specific to LoRA adaptation capacity or rank tuning.

tensor_parallel_size

Why it fails: Tensor parallel size defines how model layers are split across multiple GPUs for distributed execution. While crucial for large models, it is a parallelization strategy rather than a PEFT-specific parameter for controlling adapter capacity.

global_batch_size

Why it fails: Global batch size controls the total number of samples processed across all data-parallel ranks before performing a weight update. It influences convergence and throughput, but it does not dictate the structural adaptation capacity of LoRA adapters.

Increase the tensor parallelism degree to 8 and reduce pipeline parallelism to 1.

Why it fails: Increasing tensor parallelism to 8 would require more communication and may not fit the model on 8 GPUs with PP=1, as each GPU would hold a larger slice. It could also reduce per-GPU memory but increase communication overhead, potentially lowering throughput. This change does not directly address the low GPU utilization from small micro-batch size and may introduce other bottlenecks.

Enable activation recomputation (gradient checkpointing) and increase micro-batch size to 8.

Why it fails: Activation recomputation reduces memory usage, potentially allowing a larger micro-batch, but it adds computational overhead by recomputing activations during the backward pass. While it might enable a larger micro-batch, the net throughput gain is uncertain and often lower than using gradient accumulation. Moreover, the engineer already hit OOM at micro-batch 4, so recomputation might not free enough memory to reach 8.

Reduce the micro-batch size to 2 and double the number of GPUs to 16.

Why it fails: Reducing micro-batch size further would worsen GPU utilization and throughput. Adding more GPUs increases parallelism but also communication overhead and does not solve the underlying issue of small micro-batch. This change is costly and unlikely to improve throughput efficiently. It also requires reconfiguring the parallel strategy, which may not be feasible.

Switch to fully sharded data parallelism and reduce the global batch size to match the number of GPUs.

Why it fails: Fully sharded data parallelism partitions optimizer states, gradients, and parameters, which reduces memory per GPU but adds communication and can lower throughput for a 13B model that already fits comfortably. Reducing the global batch size also changes the training dynamics and is not required to avoid out-of-memory errors in this scenario.

Enable activation recomputation for every transformer layer and increase the number of pipeline stages to eight.

Why it fails: Activation recomputation saves memory by discarding activations and recomputing them during the backward pass, but it adds compute overhead and lowers throughput. Setting pipeline stages equal to the number of GPUs creates a deep pipeline with significant bubble time and communication cost, which is not the right throughput optimization for this model size.

Enable tensor parallelism and pipeline parallelism within the node, then increase micro-batch size until memory is nearly full.

Why it fails: Tensor and pipeline parallelism are model-parallel strategies that split layers and tensors across GPUs. They reduce per-GPU memory, but they also introduce communication overhead and do not directly maximize throughput for a 13B model that already fits on an 80GB GPU. Increasing micro-batch size after enabling them can cause memory pressure rather than a controlled throughput gain.

Increase the LoRA rank to match the hidden dimension of the model so the adapter can represent any weight update

Why it fails: Raising the rank to the hidden dimension defeats the low-rank premise and reintroduces parameter and optimizer cost comparable to full fine-tuning. It also removes the regularization benefit of a small rank and can encourage overfitting on modest datasets. The goal of learning task behavior without disturbing base weights is met by freezing plus adapters, not by inflating the rank.

Keep the base weights trainable but apply a very small global learning rate to all parameters

Why it fails: Leaving base weights trainable means they will drift during training, which contradicts the requirement to avoid disturbing them. A small learning rate merely slows the drift; it does not prevent it, and optimizer state must still be allocated for every parameter. This is full fine-tuning with a low rate, not an adapter-based approach.

Disable the distributed optimizer and replicate Adam state on every rank to simplify adapter updates

Why it fails: Replicating optimizer state on every rank raises per-GPU memory rather than lowering it and has no bearing on whether base weights stay frozen. Adapter training benefits from the distributed optimizer because it shards the smaller optimizer state across ranks. Disabling it works against the efficiency that makes adapter tuning attractive.

The entire embedding layer matrix

Why it fails: Modifying the entire embedding layer increases the number of trainable parameters significantly, which contradicts the primary goal of LoRA. Efficient fine-tuning relies on low-rank updates rather than full parameter re-training, as updating embeddings would require substantial memory and compute resources that LoRA specifically aims to bypass during training.

The feed-forward network activation functions

Why it fails: While feed-forward layers are part of the transformer, LoRA specifically targets the weight matrices of the attention mechanism. Modifying activation functions would not provide the rank decomposition benefit necessary to reduce parameter space, nor would it align with the standard implementation of LoRA as defined in current research literature.

The entire decoder hidden state output

Why it fails: Updating the decoder hidden states directly would lead to catastrophic forgetting and would not constitute a parameter-efficient fine-tuning method. LoRA functions by freezing the original weights and introducing adapters, whereas modifying the output directly would interfere with the fundamental inference mechanics of the pre-trained transformer model architecture.

To increase the training speed by using integer-only arithmetic

Why it fails: While quantization reduces memory, the training process still requires dequantization to higher precision (e.g., bfloat16) for the forward pass, which consumes additional cycles. The primary benefit is memory efficiency, not speed. Integer-only arithmetic is not the core reason for NF4; rather, it is about representational precision for weights.

To improve the convergence speed of the optimizer

Why it fails: NF4 does not inherently speed up the convergence of the optimizer. Optimizer states typically require 32-bit precision to maintain numerical stability during weight updates. While QLoRA saves memory on weights, the convergence characteristics are determined by the learning rate schedule and the quality of the gradients, not the weight precision.

To enable training without the need for gradient accumulation

Why it fails: Gradient accumulation is a technique used to simulate larger batch sizes despite memory constraints. NF4 does not replace this; in fact, one might still use gradient accumulation alongside QLoRA. The primary goal of NF4 is weight compression to fit models into memory, not to eliminate the need for batch size management.

Applying aggressive token truncation to all training samples

Why it fails: Aggressive truncation removes critical context or instructions from the input, causing the model to learn incomplete patterns. This destroys the utility of the training samples, as the model cannot associate inputs with the correct outputs if the information is cut off arbitrarily, leading to poor performance and broken responses.

Converting all text into numerical vectors before training

Why it fails: Text is processed through tokenizers, but manually converting to vectors is not part of the standard fine-tuning preparation pipeline. Modern frameworks handle tokenization, embedding lookup, and tensor creation automatically. Handling vectors manually would bypass the model's embedding layer, rendering the pre-trained weights incompatible with the fine-tuning process.

Increasing the learning rate by a factor of 100

Why it fails: A massive increase in the learning rate will cause the model's weights to explode or diverge rapidly. Fine-tuning requires delicate learning rates, often smaller than the original pre-training rate, to preserve the pre-trained knowledge while adapting to new instructions. A high learning rate causes catastrophic forgetting of general information.

The model becomes unable to produce responses in the desired language

Why it fails: While language loss can occur, it is a symptom of broader knowledge degradation, not the primary definition. Catastrophic forgetting refers to the loss of the model's learned representation of facts and reasoning capabilities, not just linguistic fluency. The loss is more systemic and affects the model's general logical reasoning.

The model's inference speed increases significantly

Why it fails: Catastrophic forgetting is an accuracy and capability issue, not a performance or latency issue. It has no direct impact on inference speed. Inference latency is determined by model size, hardware architecture, and quantization, whereas the quality of the model's outputs is determined by the weights learned during fine-tuning.

The model begins to output only empty strings

Why it fails: Outputting empty strings is typically a sign of a logic error or a failure in the generation loop, not catastrophic forgetting. A model experiencing catastrophic forgetting will still produce text, but the quality of that text will be significantly lower in terms of factual accuracy, logical consistency, and task adherence.

The number of CPU cores available for data loading

Why it fails: CPU cores impact data pre-processing speed, not GPU VRAM consumption. While fast CPU processing ensures the GPU is constantly fed data, it does not alleviate the memory constraints on the GPU itself. Bottlenecks in data loading cause under-utilization of the GPU, but they do not cause OOM errors during training.

The version of the operating system kernel

Why it fails: The kernel version does not directly impact the VRAM footprint of an LLM. While drivers and CUDA versions are critical for compatibility and performance, they do not dictate how much memory a specific model architecture consumes. Fine-tuning memory is strictly determined by model configuration, optimizer settings, and the batch size.

It sets the total number of layers that are trainable

Why it fails: The rank parameter does not control the number of layers that are trainable. That is determined by the 'target_modules' configuration. The rank specifies the internal dimension of the low-rank adaptation matrices added to the existing layers, not the depth or the selection of the layers themselves.

It dictates the number of epochs the model will train

Why it fails: Epochs are controlled by the training loop configuration or the trainer object, not the LoRA rank parameter. Setting the rank has no impact on the number of times the model iterates over the dataset. It only affects the capacity of the model to learn from the provided training data.

It determines the learning rate for the adapter weights

Why it fails: The learning rate is a separate hyperparameter defined in the training configuration. The rank is a architectural parameter that defines the capacity of the adapters. While they both influence learning, they are distinct entities. Increasing the rank does not automatically adjust the learning rate for the training process.

It speeds up the training process by calculating gradients in parallel

Why it fails: Gradient checkpointing actually increases the total computation time because it forces the system to recompute activations during the backward pass. It does not provide parallelization benefits; rather, it is a memory-saving technique that intentionally incurs a performance penalty in terms of time to allow for larger memory capacity.

It automatically adjusts the learning rate for each layer

Why it fails: Gradient checkpointing has no impact on the learning rate or the optimization algorithm. It is strictly a memory management technique for the training graph. Adaptive learning rates are managed by the optimizer, while gradient checkpointing only affects how the computational graph is stored in the GPU's memory.

It prevents the model from using the CPU during training

Why it fails: Gradient checkpointing does not isolate the model from the CPU. Data loading and CPU-GPU transfer still occur as usual. The primary goal is to reduce the VRAM consumption required for the activations generated throughout the model's layers, which is independent of the communication between CPU and GPU hardware.

1

Why it fails: A batch size of 1 is only the per-device size. Because gradient accumulation is set to 16, the effective batch size is significantly larger. The gradient is accumulated over 16 steps before updating the model weights, which is a standard practice to improve training stability when memory is limited.

15

Why it fails: The math for effective batch size is multiplication, not addition or subtraction. Adding 1 and 16 incorrectly assumes an offset in the accumulation process. The effective batch size is defined as the product of the per-device size and the accumulation steps, which is critical for understanding the training dynamics.

17

Why it fails: This result stems from incorrectly adding the two parameters. In deep learning frameworks, these parameters are multiplied to determine how many total samples contribute to each weight update. Calculating this value incorrectly can lead to misleading assumptions about the training throughput and the convergence behavior of the model during fine-tuning.

To increase the number of tokens processed in the output

Why it fails: The system prompt is not a mechanism for controlling the output token count. That is handled by max_new_tokens or similar inference parameters. The system prompt serves to define the model's role and behavior, not to artificially extend or restrict the length of the generated text response in any way.

To replace the need for domain-specific fine-tuning entirely

Why it fails: While a well-crafted system prompt can guide behavior, it cannot replace the deep, knowledge-based adaptation provided by fine-tuning. A system prompt can tell a model how to behave, but it cannot impart new knowledge or specialized task proficiency that requires weights to be updated during the fine-tuning process.

To compress the training dataset size

Why it fails: The system prompt has no relation to dataset size. It is part of the inference-time or fine-tuning-time input template. It does not compress, remove, or modify the underlying training data used during the fine-tuning process; rather, it is a fixed instruction used to shape the model's responses to user requests.