Generative AI Leader Fundamentals of Generative AI Practice Question
A data scientist is fine-tuning a large language model using Vertex AI. The training job fails with an out-of-memory error. Which action should they take to resolve this issue?
⚠ Common exam trap
Google often tests the misconception that out-of-memory errors are solved by upgrading hardware (e.g., TPU or larger model) rather than adjusting the batch size, which is the simplest and most direct fix.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Reduce the batch size
Reducing the batch size decreases the memory footprint per training step, allowing the model to fit within the available GPU or TPU memory. Out-of-memory errors during fine-tuning on Vertex AI typically occur when the batch size is too large for the allocated accelerator memory, and lowering it directly resolves the issue without changing the model architecture or hardware.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Change the accelerator to TPU
Why it's wrong here
Swapping the accelerator to a TPU changes the compute backend but not the memory footprint of the model, batch size or optimiser states, so the out-of-memory error persists unless memory per device increases. TPUs are tempting for cost-effective large-scale training, and would be correct when throughput, not memory capacity, is the bottleneck.
- ✗
Use a larger model
Why it's wrong here
A larger model increases parameter count and activation memory, worsening the out-of-memory condition during fine-tuning. Scaling up is tempting because larger models often improve accuracy, and would be the right move when quality is the constraint and accelerator memory is sufficient.
- ✗
Increase the batch size
Why it's wrong here
Larger batches raise activation-memory demand per step, worsening the out-of-memory failure rather than relieving it. Increasing batch size is a throughput-tuning lever, correct when GPU memory is underused and training is stable; here memory is already exhausted, so the batch must shrink.
- ✓
Reduce the batch size
Why this is correct
Out-of-memory during fine-tuning stems from the activation and gradient tensors held per training step. Reducing the batch size shrinks those tensors proportionally, lowering peak GPU memory below the accelerator's limit so the Vertex AI job can complete without changing the model architecture.
Visual reference
Go deeper
Related to this question
About these practice questions
This Generative AI Leader question is part of Courseiva's 1,008-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This Generative AI Leader practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the Generative AI Leader exam.