Generative AI Leader Fundamentals of Generative AI Practice Question
A company is deploying a chatbot that uses a foundation model. They want to minimize latency for user queries. Which action is most effective?
⚠ Common exam trap
Google Cloud often tests the misconception that 'bigger is better' for performance, but in latency-constrained scenarios, model size is inversely related to speed, and candidates may overlook distillation as a standard optimization technique.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use a smaller distilled model
Distilled models are smaller, faster versions of larger foundation models, trained to mimic their behavior while requiring fewer computational resources. This directly reduces inference latency because fewer parameters mean faster forward passes through the network, which is critical for real-time chatbot responses.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Use a larger model with more parameters
Why it's wrong here
Larger parameter counts increase per-token compute, raising latency rather than reducing it. This is tempting because larger models often give higher quality, which teams prioritise, but latency-sensitive chatbots need smaller or distilled models, or streaming and caching, to cut response time.
- ✗
Disable safety filters
Why it's wrong here
Safety filters run on outputs after generation, so disabling them removes a post-processing check rather than shortening the model's inference path; latency barely changes. It is tempting because filters do add some overhead, and disabling them is legitimate when throughput matters and content risk is controlled.
- ✗
Increase the number of tokens
Why it's wrong here
More tokens lengthens the generated sequence, and each additional token requires another forward pass, so latency rises rather than falls. Increasing token limits is the right lever when responses are being truncated mid-sentence and completeness matters more than speed.
- ✓
Use a smaller distilled model
Why this is correct
A smaller distilled model has fewer parameters and lower inference compute per token, directly cutting response latency. Distillation transfers the larger model's behaviour into a compact architecture, satisfying the latency-minimisation constraint while retaining acceptable quality for chatbot queries.
Go deeper
Related to this question
About these practice questions
Courseiva writes every Generative AI Leader question from scratch — 1,008 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This Generative AI Leader practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the Generative AI Leader exam.