A company is using Amazon SageMaker to deploy a model for real-time inference. The model requires 500 MB of memory and has a latency requirement of 100 ms. The endpoint is receiving 10 requests per second. Which instance type should be chosen for cost-effectiveness?
Adequate memory and cost-effective.
Why this answer
Ml.m5.large (Option C). This instance type provides 2 vCPU and 8 GB memory, which is more than sufficient for the 500 MB memory requirement. It also offers consistent performance suitable for real-time inference with 10 requests per second and 100 ms latency.
Option A (ml.c5.xlarge) has 4 vCPU and 8 GB, which is over-provisioned and more expensive. Option B (ml.t2.medium) has only 4 GB memory but uses burstable CPU, which may cause latency spikes. Option D (ml.p3.2xlarge) is GPU-optimized and significantly more expensive, making it unsuitable for a CPU-bound workload.
Therefore, ml.m5.large is the most cost-effective choice.