MLA-C01 Deployment and Orchestration of ML Workflows Practice Question

A data science team wants to host 50 different models for a recommendation engine. Each model is small (under 100 MB) and traffic patterns are unpredictable. They need to minimize cost and operational overhead. Which approach should they take?

⚠ Common exam trap

AWS often tests the distinction between multi-model endpoints (for multiple independent models) and multi-container endpoints (for a single model with multiple containers), leading candidates to confuse the two and incorrectly choose option D.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Use a single multi-model endpoint (MME)

A single multi-model endpoint (MME) allows hosting multiple models (up to thousands) on the same endpoint, sharing the underlying compute instance. This minimizes cost and operational overhead for small models (under 100 MB) with unpredictable traffic, as the endpoint dynamically loads and unloads models from Amazon S3 into memory based on incoming requests, eliminating the need for separate endpoints or idle compute.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Deploy each model to its own real-time endpoint
Why it's wrong here
A real-time endpoint per model bills continuously for 50 always-on instances, which is wasteful given small artefacts and unpredictable traffic. It is tempting because endpoints offer low-latency inference, but this scenario suits a multi-model endpoint or serverless inference that scales to zero.
✗
Use SageMaker serverless inference for each model
Why it's wrong here
Serverless inference scales to zero but cold-start latency and per-model configuration still multiply across 50 endpoints, and it caps memory and container size. It tempts because serverless fits sporadic, unpredictable traffic for a single model, not a fleet needing shared hosting.
✓
Use a single multi-model endpoint (MME)
Why this is correct
A multi-model endpoint hosts many models behind one container, so 50 small models share a single endpoint rather than 50 separate ones. This directly minimises cost and operational overhead while handling unpredictable traffic through shared autoscaling.
✗
Use a single multi-container endpoint
Why it's wrong here
A multi-container endpoint runs several containers on one instance but does not scale each model independently, so unpredictable per-model traffic still forces over-provisioning. It tempts because multi-container endpoints suit tightly coupled containers invoked together, such as preprocessing plus inference pipelines.

Quick reference

AWS S3 Storage Class Comparison

Storage Class	Min Duration	Retrieval	Use Case
S3 Standard	None	Immediate	Frequently accessed data
S3 Standard-IA	30 days	Immediate	Infrequent access, rapid retrieval
S3 One Zone-IA	30 days	Immediate	Non-critical infrequent data
S3 Intelligent-Tiering	None	Immediate–hours	Unknown or changing access patterns
S3 Glacier Instant	90 days	Milliseconds	Archive with instant retrieval
S3 Glacier Flexible	90 days	Minutes–hours	Archive, flexible retrieval
S3 Glacier Deep Archive	180 days	Hours	Long-term compliance archive

Go deeper

MLA-C01 Deployment and Orchestration of ML Workflows Practice Question

Why each option matters

Option-by-option breakdown

AWS S3 Storage Class Comparison

Related to this question