1Z0-1127-25 Deploying and Managing Generative AI on OCI Practice Question
Your company uses OCI Data Science for model development and deployment. You have a generative AI model that requires dynamic batching for efficient inference. You deployed the model using the OCI Model Deployment service with a custom inference script in a Docker container. However, you notice that the batch size is fixed at 1, leading to low throughput. The model can process multiple requests together efficiently. You want to implement dynamic batching to increase throughput without significantly increasing latency for individual requests. What is the best approach?
⚠ Common exam trap
Watch out — candidates often assume OCI Model Deployment has a built-in batching feature similar to some cloud ML services, but OCI requires you to implement batching logic yourself in the custom inference script.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Implement a queuing mechanism in the inference script that collects incoming requests and processes them in batches
Dynamic batching must be implemented at the application level within the custom inference script when using OCI Model Deployment. The service does not provide built-in request batching; instead, you need to collect incoming requests in a queue and process them together in a single forward pass, which maximizes GPU utilization while controlling latency via a timeout or max batch size.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Modify the model deployment to use a larger GPU shape to handle larger batches
Why it's wrong here
A larger GPU does not automatically batch requests; the inference code must support batching.
- ✗
Enable the model deployment's built-in request batching feature
Why it's wrong here
OCI Model Deployment does not have a built-in dynamic batching feature.
- ✗
Use OCI Streaming service to buffer requests and then invoke the model in batches from a consumer
Why it's wrong here
This adds significant latency and complexity compared to in-container batching.
- ✓
Implement a queuing mechanism in the inference script that collects incoming requests and processes them in batches
Why this is correct
This is a common pattern for dynamic batching and can be done within the custom container.
Go deeper
Related to this question
About these practice questions
This 1Z0-1127-25 question is part of Courseiva's 768-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This 1Z0-1127-25 practice question is part of Courseiva's free Oracle certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the 1Z0-1127-25 exam.