Generative AI Leader Practice Question: Business Strategies for Generative AI Solutions
A company has a generative AI chatbot on Vertex AI that shows high response latency. They want to reduce latency without significantly increasing cost. Which TWO actions should they take? (Choose two.)
⚠ Common exam trap
Many exam-takers confuse reducing latency with reducing total generation time in Vertex AI, but streaming only reduces perceived latency by delivering tokens earlier, while options like reducing max_output_tokens actually cut total generation time and cost by limiting output length.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Enable streaming responses using server-sent events.
Enabling streaming responses using server-sent events (SSE) allows the chatbot to send tokens incrementally as they are generated, rather than waiting for the full response. This reduces the perceived latency for the end user, as the first token appears much sooner, even though the total generation time may remain similar. This approach directly addresses high response latency without increasing compute cost, as it does not require additional infrastructure or model changes.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Increase the min_replica_count to keep more instances always warm.
Why it's wrong here
Increasing min_replica_count keeps more instances always warm, which reduces cold start latency but increases cost because you pay for idle resources. This fails the 'without significantly increasing cost' condition.
- ✓
Enable streaming responses using server-sent events.
Why this is correct
Enabling streaming responses using server-sent events sends tokens incrementally, making the first token appear sooner. This reduces perceived latency without requiring additional infrastructure or cost.
- ✓
Reduce the max_output_tokens parameter in the model configuration.
Why this is correct
Reducing max_output_tokens decreases the total number of tokens the model generates, directly cutting generation time and cost. This is a simple configuration change with no extra infrastructure.
- ✗
Use machine types with GPUs.
Why it's wrong here
Using GPU machine types increases compute cost significantly and may not proportionally reduce latency for all models. It violates the cost constraint.
- ✗
Switch to a larger model like Gemini 1.5 Pro for better accuracy.
Why it's wrong here
Switching to a larger model like Gemini 1.5 Pro increases latency due to higher computational demands and raises cost. It does not reduce latency.
Go deeper
Related to this question
About these practice questions
This Generative AI Leader question is part of Courseiva's 683-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This Generative AI Leader practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the Generative AI Leader exam.