AI-102 Plan and manage an Azure AI solution Practice Question
You are an AI engineer at a healthcare company. The company uses Azure Cognitive Services to process medical records. They have a Computer Vision resource deployed in the East US region. Recently, they implemented a custom vision model for detecting specific anomalies in X-ray images. The model was trained using the Custom Vision portal and exported as a TensorFlow model. They deployed the model to an Azure Kubernetes Service (AKS) cluster using a Docker container. The container runs the model and exposes a REST API endpoint for inference. The endpoint is used by a web application that is also hosted in the same AKS cluster. The web application is experiencing high latency when making inference requests. The latency spikes up to 10 seconds during peak hours. The AKS cluster has autoscaling enabled based on CPU metrics. The container's resource limits are set to 1 CPU core and 2 GB memory. The model's inference time on a single image is approximately 500 ms on the development machine. The team has not changed the model or the application code recently. The number of concurrent users has increased by 50% in the last month. What should you do to reduce inference latency?
⚠ Common exam trap
Watch out — candidates often confuse horizontal scaling (adding replicas) with vertical scaling (increasing CPU cores), but in this scenario, the bottleneck is per-request CPU throughput, not request volume, so vertical scaling is the correct fix.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Increase the CPU limit for the container to 4 cores.
The inference latency is caused by CPU saturation during peak hours. The container is limited to 1 CPU core, and with a 50% increase in concurrent users, the single core becomes a bottleneck, causing inference times to spike. Increasing the CPU limit to 4 cores allows the model to process multiple requests in parallel, reducing queue wait times and overall latency.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Increase the memory limit for the container to 8 GB.
Why it's wrong here
Memory is not the bottleneck; CPU is.
- ✗
Deploy additional replicas of the container and use a load balancer.
Why it's wrong here
Adding replicas with a load balancer distributes requests across multiple identical containers, which reduces per-container CPU load but does not address the root cause: the model’s 500 ms inference time per image is a fixed per-request latency that cannot be shortened by horizontal scaling. This option is tempting because it is the standard remedy for throughput bottlenecks under increased concurrent user load, and it would be correct if the latency were caused by queueing delays from insufficient processing capacity rather than by the model’s own single-image inference duration.
- ✗
Optimize the model by quantizing it to reduce inference time.
Why it's wrong here
Inference time per image is already low; the issue is concurrency.
- ✓
Increase the CPU limit for the container to 4 cores.
Why this is correct
More CPU allows handling more concurrent requests, reducing latency.
Go deeper
Related to this question
About these practice questions
Courseiva writes every AI-102 question from scratch — 945 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-102 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-102 exam.