MLA-C01 Timed Practice Test 3 — 10 Questions

Question 1

A team receives alerts that their SageMaker endpoint latency has increased significantly. They check CloudWatch metrics and see Invocations rising, but ModelLatency remains stable. Which metric should they investigate to find the source of the increased latency?

10 minute time limit — choose an answer to begin.

Accepted Answer

OverheadLatency. OverheadLatency measures the time taken by the SageMaker infrastructure to handle requests before and after model inference, including request routing, authentication, and response processing. Since ModelLatency is stable but total endpoint latency has increased, the extra time must be in the overhead component, making OverheadLatency the correct metric to investigate.

Answer

ModelLatency

Answer

5XXError

Answer

4XXError

Question 2

An ML team is deploying a model using SageMaker. The model requires GPU inference and must be available in multiple AWS regions for low latency. The team has created a multi-model endpoint with GPU instances. After deployment, they notice high latency spikes when a new model is loaded. What is the most likely cause?

Accepted Answer

The team is using a multi-model endpoint, which loads models on demand; loading a model into GPU memory causes latency spikes.. A multi-model endpoint (MME) loads models on demand from Amazon S3 into the instance's memory. When a new model is requested and not already cached, SageMaker must download the model artifacts and load them into GPU memory, which is a time-consuming operation that causes a latency spike for the first inference request. This cold-start behavior is inherent to MMEs and explains the observed spikes.

Answer

The endpoint is configured with a single production variant, causing all traffic to overload one instance.

Answer

The endpoint is using the wrong instance type that lacks sufficient GPU memory.

Answer

The model is too large for the specified container memory, causing swap to disk.

Question 3

A company uses SageMaker endpoints with auto-scaling based on CPU utilization. During a flash sale, latency increases despite low CPU. What should be done?

Accepted Answer

Use a custom metric such as memory utilization or request count for auto-scaling. CPU utilization is a poor scaling metric for inference workloads that are I/O or memory-bound. During a flash sale, increased request concurrency can cause queuing and latency spikes even when CPU is low. Using a custom metric like request count per instance or memory utilization directly reflects the load on the inference endpoint, enabling the Application Auto Scaling target tracking policy to scale out proactively before latency degrades.

Answer

Increase the instance size

Answer

Disable auto-scaling and use a larger instance

Answer

Switch to GPU instances

Question 4

A company's SageMaker real-time endpoint is experiencing high latency under load. The CloudWatch metrics show that the ModelLatency is acceptable, but the OverheadLatency is spiking. What is the most likely cause?

Accepted Answer

The endpoint is under-provisioned with insufficient instance count.. OverheadLatency measures the time spent on infrastructure overhead (e.g., request routing, network I/O, and container startup) rather than model inference. When the endpoint is under-provisioned with too few instances, requests queue up, causing the SageMaker front-end to wait for a free worker, which directly inflates OverheadLatency while ModelLatency (pure inference time) remains unaffected.

Answer

The request payload size is too large.

Answer

The SageMaker endpoint is not in the same VPC as the client.

Answer

The model inference code is inefficient.

Question 5

A company uses SageMaker for training and inference. They have a model that retrains weekly. After each retraining, the model is evaluated on a held-out test set. If the evaluation metrics meet a threshold, the model is registered as 'Approved' in the SageMaker Model Registry. The team manually deploys the approved model to a production endpoint. They want to automate this deployment process to reduce manual errors. However, the deployment should only proceed if the new model passes a canary test in a staging environment. Which combination of AWS services should the team use to achieve this?

Accepted Answer

SageMaker Pipelines with a conditional deployment step that includes a canary test.. SageMaker Pipelines natively supports conditional execution steps, allowing you to add a canary test step that evaluates the new model in a staging environment before automatically promoting it to production. This directly addresses the requirement for automated deployment gated by a canary test, without needing external orchestration services.

Answer

AWS CodeDeploy with a blue/green deployment strategy.

Answer

AWS Lambda to deploy to staging, then automatically promote to production if staging tests pass.

Answer

Amazon EKS with a custom inference container and use ArgoCD for automated deployments.

Question 6

A data scientist needs to deploy a single ML model that will serve real-time predictions with low latency (under 10 ms) for a high-traffic web application. The model fits in memory and requires GPU acceleration. Which SageMaker inference option is MOST suitable?

Accepted Answer

Real-time endpoint on ml.g4dn instances. Real-time endpoints on GPU instances (ml.g4dn) provide low latency and GPU acceleration, ideal for high-traffic, latency-sensitive workloads.

Answer

Real-time endpoint on ml.m5 instances

Answer

Batch Transform

Answer

Serverless Inference

Question 7

A data scientist runs the exhibit AWS Glue ETL job. The job fails with a Spark stage failure error. What is the most likely cause?

Accepted Answer

The data type mapping in ApplyMapping is incorrect; "value" column contains non-numeric strings that cannot be cast to double.. The Spark stage failure error in an AWS Glue ETL job is most likely caused by a data type mismatch during the ApplyMapping transformation. When the 'value' column contains non-numeric strings that cannot be cast to double, Spark throws a stage failure because it cannot complete the required type conversion, leading to task failures and job termination.

Answer

The output path is missing.

Answer

The S3 bucket does not exist.

Answer

The job does not have enough memory.

Question 8

A data scientist has trained a binary classification model for fraud detection. The dataset is highly imbalanced (99% non-fraud, 1% fraud). After evaluation, the model shows an accuracy of 99%, but the recall for fraud cases is only 10%. Which metric should the data scientist prioritize to improve the model's performance for fraud detection?

Accepted Answer

F1-score. F1-score balances precision and recall, making it more informative than accuracy for imbalanced datasets. AUC-ROC is also used but F1 directly addresses the trade-off between false positives and false negatives. Precision alone does not capture recall, and Log loss does not directly indicate recall improvement.

Answer

Log loss

Answer

Precision

Answer

Area under the ROC curve (AUC-ROC)

Question 9

A team uses SageMaker Clarify to monitor bias drift on a deployed model. They have defined a baseline with training data and set up a monitoring schedule. After one month, they receive a violation report indicating that the post-training metrics have deviated from the baseline. What does this violation indicate?

Accepted Answer

The model's predictions relative to sensitive attributes have shifted compared to the training baseline. SageMaker Clarify bias drift monitoring compares predicted outcomes (post-training) against the baseline to detect changes in fairness metrics like disparate impact. It does not measure prediction accuracy or data quality.

Answer

The SHAP values for features have changed

Answer

The model's predictions are becoming less accurate

Answer

The distribution of input features has changed

Question 10

A data scientist creates a feature group as shown in the exhibit. When ingesting data with an 'age' column of integer values, the ingestion fails. What is the most likely cause?

Accepted Answer

The `age` feature type should be `Integral`, not `String`.. The feature group definition specifies the 'age' column as a `String` type, but the ingested data contains integer values. Amazon SageMaker Feature Store requires that the data types of ingested records match the schema defined in the feature group. When a mismatch occurs, such as providing an integer for a string field, the ingestion fails with a type conversion error.

Answer

The role does not have permissions to write to the feature store.

Answer

The `OnlineStoreConfig` must include a `SecurityConfig`.

Answer

The `EventTimeFeatureName` is incorrectly spelled.