A machine learning team is deploying a real-time inference endpoint for a fraud detection model using Amazon SageMaker. The model is a LightGBM classifier trained on 1 GB of tabular data. The endpoint must respond within 100 ms for 99% of requests, with a throughput of 10 requests per second. During load testing, the team observes that the 99th percentile latency is 250 ms and the endpoint CPU utilization is consistently above 90%. The team has already selected an ml.c5.xlarge instance with auto scaling enabled. Which combination of actions should the team take to meet the latency requirement? (Choose 3.)

Question

Accepted Answer

Upgrade the instance type to ml.c5.2xlarge to increase CPU resources per instance.. (upgrading to ml.c5.2xlarge) provides more CPU resources per instance, reducing CPU utilization and thus latency. Option B (reducing the number of trees in the LightGBM model) decreases the computational complexity of inference, directly lowering inference time. Option C (enabling SageMaker's data compression for endpoint input payloads) reduces the data transfer size, which can lower I/O overhead and network latency. Option D (switching to SageMaker Batch Transform) is unsuitable because it is not designed for real-time inference and would not meet the low-latency requirement. Together, options A, B, and C address the latency issue by improving compute capacity, reducing model complexity, and minimizing data transfer time.

Answer

Switch to using SageMaker Batch Transform instead of a real-time endpoint.

MLS-C01 Modeling Practice Question

Why each option matters

Option-by-option breakdown

Question Discussion