easyMultiple Select

Generative AI Leader Practice Question: A team is deciding between using fine-tuning and…

A team is deciding between using fine-tuning and in-context learning for a document classification task. They have 500 labeled examples and need low latency. Which TWO statements are accurate?

⚠ Common exam trap

Google often tests the misconception that in-context learning always has lower latency than fine-tuning, but the trap is that latency depends on prompt length and model architecture, not just the absence of a training step.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Fine-tuning generally improves accuracy more than in-context learning when sufficient labeled data is available

Option D is correct because fine-tuning updates the model's weights on the 500 labeled examples, allowing it to learn task-specific patterns and typically achieve higher accuracy than in-context learning when a sufficient amount of labeled data is available. Option E is correct because in-context learning places examples directly in the prompt at inference time, requiring no gradient updates or training step, so it can be used immediately with a pretrained model. Option A is incorrect because in-context learning increases prompt length and thus inference latency, so it does not always have lower latency than fine-tuning. Option B is incorrect because in-context learning works with any model context window and is not limited to windows smaller than 1000 tokens. Option C is incorrect because fine-tuning still requires a validation dataset to tune hyperparameters and monitor for overfitting.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
In-context learning always has lower latency than fine-tuning
Why it's wrong here
Latency depends on prompt length and output tokens, not the technique itself; long in-context examples can exceed fine-tuning inference time. It is tempting because in-context learning avoids a training step, making it the right choice when examples are few and latency is not the binding constraint.
✗
In-context learning can only be used with models that have a context window smaller than 1000 tokens
Why it's wrong here
In-context learning requires a context window large enough to hold the examples plus input; larger windows expand its usefulness rather than restrict it. It is tempting because small windows do limit how many examples fit, making in-context learning unsuitable for very long prompts.
✗
Fine-tuning eliminates the need for a validation dataset
Why it's wrong here
Fine-tuning still requires a held-out validation set to detect overfitting and select checkpoints; the 500 labelled examples must be split. It is tempting because training consumes the labelled data, but validation remains essential whenever gradient updates are applied to a model.
✓
Fine-tuning generally improves accuracy more than in-context learning when sufficient labeled data is available
Why this is correct
Fine-tuning adjusts the model's weights on the 500 labelled examples, embedding task-specific patterns directly into parameters rather than supplying them at inference. This yields higher accuracy than in-context learning once labelled data is sufficient, satisfying the scenario's classification goal, though it does not address the low-latency constraint.
✓
In-context learning requires no training step and can be used immediately
Why this is correct
In-context learning places task examples directly in the prompt, so the model adapts without any weight updates or training pipeline. This satisfies the low-latency constraint, since inference begins immediately with no fine-tuning job to run or deploy.

Go deeper

Generative AI Leader Practice Question: A team is deciding between using fine-tuning and…

Why each option matters

Option-by-option breakdown

Related to this question