A team is deploying a generative AI model for a real-time customer-facing application. They need to balance cost and latency. Which deployment strategy is MOST suitable?
Microservices with streaming and async queues reduce perceived latency and handle variable load efficiently.
Why this answer
Option D is correct because AI microservices with streaming responses and async processing queues decouple inference from the request lifecycle, allowing the system to handle variable loads efficiently while maintaining low latency for real-time interactions. This architecture balances cost by scaling only the necessary components (e.g., GPU-backed inference services) and uses streaming (e.g., Server-Sent Events or WebSockets) to deliver partial results, reducing perceived latency for the customer.
Exam trap
Cisco often tests the misconception that serverless functions (Option A) are always the cheapest and fastest option, but they ignore cold-start latency and the overhead of monolithic orchestration in real-time AI workloads.
How to eliminate wrong answers
Option A is wrong because a monolithic API with serverless functions introduces cold-start latency and tight coupling, which is unsuitable for real-time customer-facing applications where consistent sub-second response times are critical. Option B is wrong because edge deployment on user devices requires significant on-device compute resources, model compression, and frequent updates, which increases deployment complexity and cost, and may not be feasible for large generative models. Option C is wrong because batch processing with synchronous requests is designed for high-throughput, non-real-time workloads (e.g., nightly report generation) and would force users to wait for batch completion, violating the real-time requirement.