1Z0-1127-25 Deploying and Managing Generative AI on OCI Practice Question
A generative AI model deployed on OCI Model Deployment is experiencing high tail latency. The model is a large language model that processes variable-length input sequences. Profiling shows that inference time varies significantly: short inputs (100 tokens) take 100ms, while long inputs (2000 tokens) take 2 seconds. The application requires consistent low latency (<500ms) for most requests. You want to reduce the variance in inference time without major changes to the model architecture. Which technique should you apply?
⚠ Common exam trap
Test-takers frequently confuse horizontal scaling (Option B) with latency variance reduction, but scaling replicas does not address the root cause of variable inference time due to sequence length differences.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Implement dynamic batching that groups requests of similar lengths together before inference
Dynamic batching groups requests of similar input lengths together, which reduces the variance in inference time by ensuring that each batch processes tokens of comparable size. This minimizes the padding overhead and keeps the per-request latency more predictable, directly addressing the high tail latency caused by variable-length sequences without altering the model architecture.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Implement dynamic batching that groups requests of similar lengths together before inference
Why this is correct
Grouping by length reduces the overhead from padding and stabilizes inference time.
- ✗
Increase the number of replicas to distribute the load evenly
Why it's wrong here
More replicas improve throughput but not per-request latency variance.
- ✗
Reduce the model size by removing layers or using a smaller version
Why it's wrong here
This changes model accuracy and is not desired.
- ✗
Deploy multiple model endpoints for different length ranges and route requests accordingly
Why it's wrong here
This adds operational complexity and may not reduce tail latency as each endpoint still faces variable lengths.
Go deeper
Related to this question
About these practice questions
This 1Z0-1127-25 question is part of Courseiva's 768-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This 1Z0-1127-25 practice question is part of Courseiva's free Oracle certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the 1Z0-1127-25 exam.