AIF-C01 Fundamentals of Generative AI Practice Question
A financial services company is deploying a generative AI model on Amazon SageMaker for real-time fraud detection. The model, a fine-tuned Llama 2 7B, must respond to transaction requests within 500 milliseconds. The team has deployed the model using a SageMaker real-time endpoint with a single ml.g5.2xlarge instance. During load testing, the endpoint achieves an average latency of 450 ms at 10 requests per second (RPS), but the latency spikes to over 2 seconds at 20 RPS. The team needs to maintain sub-500 ms latency at up to 50 RPS. The model is too large to fit on a single GPU, so they are using CPU instances. They considered using a larger instance type but want to minimize cost. What should the team do to meet the latency requirement cost-effectively?
⚠ Common exam trap
Test-takers frequently assume a larger single instance (Option A) is the simplest solution, but they overlook the cost-efficiency and scalability benefits of horizontal scaling with a multi-model endpoint, which is specifically designed for high-throughput, low-latency inference with models that don't fit on a single GPU.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use a SageMaker multi-model endpoint with multiple ml.g5.xlarge instances and auto scaling
A SageMaker multi-model endpoint (MME) allows multiple model replicas to be hosted on a fleet of instances, enabling horizontal scaling to handle increased throughput. By using multiple ml.g5.xlarge instances with auto scaling, the team can distribute the 50 RPS load across several instances, keeping per-instance latency low while minimizing cost compared to a single larger instance. This approach also leverages the fact that the model is too large for a single GPU but can be efficiently served on CPU instances with proper load distribution.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Upgrade to a single ml.g5.4xlarge instance
Why it's wrong here
A larger instance may still not handle 50 RPS within latency, and cost is higher.
- ✗
Attach an Amazon Elastic Inference accelerator to the existing instance
Why it's wrong here
Elastic Inference is deprecated and not cost-effective for this scenario.
- ✓
Use a SageMaker multi-model endpoint with multiple ml.g5.xlarge instances and auto scaling
Why this is correct
Distributing load across smaller instances reduces cost and meets latency via scaling.
- ✗
Use SageMaker Serverless Inference to automatically scale
Why it's wrong here
Serverless Inference can have cold start latency exceeding 500 ms.
Go deeper
Related to this question
About these practice questions
One of 619 original AIF-C01 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AIF-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AIF-C01 exam.