NCP-GENL Data Preparation Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✓

  Include a diverse set of instruction types and domains in the training data rather than repeating a few templates.

  Why this is correct

  Diversity in instruction types and domains encourages the model to learn the general pattern of following instructions rather than memorizing specific templates. If the training data is dominated by a few templates, the model may overfit to those formats and fail on novel instructions. A broad distribution of tasks improves zero-shot and few-shot generalization.

• ✗

  Increase the learning rate significantly to force the model to escape memorization of individual examples.

  Why it's wrong here

  A higher learning rate does not prevent memorization; it can destabilize training and cause divergence. Memorization is primarily controlled by data diversity, dataset size, and regularization, not by learning rate alone. In fact, an excessively high learning rate may prevent the model from learning useful patterns at all. Data preparation practices, not optimizer settings, are the focus here.

• ✗

  Duplicate the most common instruction-response pairs to reinforce the desired behavior.

  Why it's wrong here

  Duplicating common pairs biases the model toward those specific examples and increases the risk of memorization. It also skews the loss toward overrepresented patterns, reducing generalization to other instructions. For SFT, balanced and diverse data is preferred, and duplicates should be removed rather than added. This practice contradicts the goal of generalization.

• ✗

  Use only English instructions to simplify tokenization and avoid multilingual complexity.

  Why it's wrong here

  Restricting to English reduces diversity and may harm generalization if the model is expected to handle other languages or code. It does not address memorization within English. A diverse instruction set, including multiple languages if relevant, is better for generalization. Simplifying tokenization is not a valid reason to limit the training distribution in a way that could degrade downstream performance.

• ✓

  Split the dataset into training, validation, and test sets with no overlapping instructions or responses.

  Why this is correct

  Separating data into disjoint splits prevents the model from seeing validation or test examples during training. Without this, evaluation metrics would be optimistically biased and the model could memorize specific examples. For SFT, it is important that instructions and responses in the validation set are not present in training, so that performance reflects generalization to new instructions.