Fundamentals of Large Language Models →mediumMultiple ChoiceObjective-mapped

1Z0-1127-25 Fundamentals of Large Language Models Practice Question

During fine-tuning of a Cohere model on OCI Data Science, the loss curve shows a sharp spike after epoch 3. What is the most appropriate action?

⚠ Common exam trap

Oracle often tests the distinction between gradient explosion (sharp spikes) and learning rate divergence (gradual increase), leading candidates to incorrectly choose reducing the learning rate instead of gradient clipping.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Gradient clipping.

A sharp spike in the loss curve after epoch 3 during fine-tuning indicates a gradient explosion, where the gradients become excessively large and destabilize the model's weights. Gradient clipping is the most appropriate action because it directly caps the gradient norm (e.g., using `max_grad_norm=1.0` in Cohere's fine-tuning API) to prevent these spikes, ensuring stable training without altering the learning dynamics.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✓
Gradient clipping.
Why this is correct
Gradient clipping limits gradient values, preventing explosion and stabilizing training.
✗
Reduce learning rate.
Why it's wrong here
Reducing LR can help but is a less targeted fix; gradient clipping is more effective for spikes.
✗
Add more training data.
Why it's wrong here
Adding data may not address the immediate gradient explosion; it could even introduce more variance.
✗
Increase batch size.
Why it's wrong here
Increasing batch size may smooth gradients but does not prevent explosive spikes.

Go deeper

1Z0-1127-25 Fundamentals of Large Language Models Practice Question

Why each option matters

Option-by-option breakdown

Related to this question