MLA-C01 Quick Quiz — 10 Questions

Question 1

A team uses SageMaker Clarify to monitor bias drift in production. They schedule weekly analysis. After a month, Clarify reports a significant increase in a bias metric. What should the team do first?

Accepted Answer

Review the analysis report to understand which feature and segment contributed to the drift.. Reviewing the report helps understand which feature and segment contributed to the drift. Option A is premature without understanding the cause. Option B is ignoring the issue. Option C does not address the drift source.

Answer

Disable the bias monitor because the metric may be noisy.

Answer

Immediately retrain the model with a balanced dataset.

Answer

Increase the frequency of analysis to daily.

Question 2

A team is deploying a machine learning model for real-time fraud detection. The model must have inference latency under 10 ms and handle up to 1000 requests per second. The model is a gradient boosting model using XGBoost. Which SageMaker hosting configuration is MOST cost-effective while meeting the requirements?

Accepted Answer

Use a SageMaker Multi-Model Endpoint (MME) on an ml.c5.4xlarge instance with auto scaling. The most cost-effective configuration is using a Multi-Model Endpoint (MME) on an ml.c5.4xlarge instance with auto scaling. The ml.c5.4xlarge instance provides sufficient compute (16 vCPUs) to achieve under 10 ms inference latency and handle 1000 requests per second. Auto scaling ensures the endpoint adapts to traffic, minimizing cost while meeting demand. Although MME is typically designed for multiple models, it can also host a single model; the cost advantage here comes from the right-sized instance and scaling, not from the multi-model feature. A single ml.c5.xlarge instance (option C) lacks the vCPU capacity to handle 1000 req/s at sub-10ms latency. Deploying separate endpoints on ml.m5.large instances (option D) would require many instances, increasing cost. Batch Transform (option A) is for batch inference, not real-time.

Answer

Use SageMaker Batch Transform with multiple instances

Answer

Deploy on a single ml.c5.xlarge instance with a real-time endpoint

Answer

Deploy separate real-time endpoints for each model on ml.m5.large instances

Question 3

A company wants to deploy a PyTorch model on SageMaker for real-time inference. Which two steps are required? (Select TWO.)

Accepted Answer

Package the model artifacts into a tar.gz file.. SageMaker requires model artifacts to be packaged as a single tar.gz file (containing the model weights, serialized PyTorch model, and any dependencies) for deployment. This compressed archive is uploaded to S3 and referenced when creating the model object for real-time inference.

Answer

Upload the training data to an S3 bucket.

Answer

Register the model in the SageMaker Model Registry.

Answer

Set up a SageMaker Notebook instance.

Question 4

A machine learning team is preparing data for a binary classification model. The target variable has a severe class imbalance (95% negative, 5% positive). The team wants to train a model that maximizes recall on the positive class while keeping training time manageable. Which approach should they use?

Accepted Answer

Use SMOTE to generate synthetic samples for the positive class. SMOTE (Synthetic Minority Oversampling Technique) creates synthetic samples of the minority class, increasing its representation without simply duplicating data. This often improves recall without drastically increasing training time compared to other methods.

Answer

Use class weights inversely proportional to class frequencies

Answer

Oversample the positive class by simply duplicating existing records

Answer

Undersample the majority class to match the minority class size

Question 5

A media company uses SageMaker to host a real-time video recommendation model. The model is deployed on a single ml.c5.xlarge endpoint. During a major live event, traffic surges to 10 times the normal load, and the endpoint becomes unresponsive, causing high latency and errors. The team had set up an Application Auto Scaling target tracking policy based on CPU utilization with a target of 70%. However, scaling did not trigger quickly enough. After the event, the team reviews CloudWatch metrics and notices that CPU utilization never exceeded 70% during the surge, but memory utilization peaked at 95%. The model is memory-bound. The team wants to ensure the endpoint scales automatically before performance degrades during future events. What should the team do?

Accepted Answer

Change the target tracking metric to memory utilization and set a target of 70%. The model is memory-bound, and the current CPU-based target tracking policy failed to trigger scaling since CPU utilization never exceeded 70% during the surge. By switching to a memory utilization metric with a target of 70%, scaling will activate based on the actual resource constraint (memory), preventing performance degradation before the endpoint becomes unresponsive.

Answer

Increase the target CPU utilization to 90% so that scaling triggers at higher load

Answer

Change the endpoint instance type to ml.c5.4xlarge to provide more memory per instance

Answer

Create a scheduled scaling policy to add instances during the known event time

Question 6

A company uses Amazon SageMaker to train and deploy a machine learning model. After deployment, they notice that the model's accuracy drops significantly over time due to changes in the underlying data distribution. Which monitoring solution should they implement to detect this issue automatically?

Accepted Answer

Set up Amazon SageMaker Model Monitor with data quality monitoring.. Amazon SageMaker Model Monitor with data quality monitoring is the correct solution because it automatically detects deviations in the input data distribution compared to a baseline, which directly addresses the problem of model accuracy degradation due to data drift. It continuously monitors the statistical properties of inference requests and alerts when drift is detected, enabling proactive retraining.

Answer

Configure AWS Config rules to check the model accuracy metric.

Answer

Use AWS CloudTrail to monitor changes to the model's S3 bucket.

Answer

Enable Amazon CloudWatch Logs on the endpoint and set alarms on inference latency.

Question 7

A machine learning engineer is deploying a model using a SageMaker endpoint and needs to ensure that the model artifacts are encrypted at rest using a customer-managed KMS key. Which configuration should they set?

Accepted Answer

Set the KMS key when creating the model using the CreateModel API. The `CreateModel` API in SageMaker accepts a `ModelKmsKeyId` parameter that specifies a customer-managed KMS key for encrypting the model artifacts at rest. This key is used when SageMaker copies the artifacts from S3 to the inference instance's Amazon EBS volume, ensuring encryption at rest. The other options either apply to different resources or do not control the encryption of the model artifacts themselves.

Answer

Set the KMS key in the endpoint configuration's ProductionVariant

Answer

Enable default encryption on the S3 bucket containing the model artifacts

Answer

Use SageMaker Studio's KMS integration

Question 8

A company wants to reduce costs for a production SageMaker endpoint that has predictable traffic patterns. They have purchased a Savings Plan. What additional step can they take to further optimize costs while maintaining performance?

Accepted Answer

Use SageMaker Inference Recommender to right-size the endpoint. SageMaker Inference Recommender provides instance type and configuration recommendations to right-size endpoints, balancing cost and performance. It is the appropriate tool for cost optimization beyond a Savings Plan.

Answer

Reduce the number of instances to one, regardless of load

Answer

Switch from real-time to batch inference

Answer

Disable auto-scaling

Question 9

A machine learning engineer is developing a text classification model using Amazon SageMaker. The dataset consists of 1 million customer reviews, with labels indicating sentiment (positive, negative, neutral). The engineer uses a pre-trained BERT model from the Hugging Face Model Hub and fine-tunes it on the dataset using SageMaker's Hugging Face estimator with a ml.p3.2xlarge instance. After 2 hours of training, the training job fails with a 'ResourceExhaustedError: CUDA out of memory' error. The error occurs during the forward pass of the first epoch. The engineer confirms that the batch size is set to 32, the maximum sequence length is 512 tokens, and the dataset is stored in a S3 bucket in the same AWS region. The engineer needs to complete fine-tuning without increasing instance costs. Which course of action should the engineer take?

Accepted Answer

Reduce the batch size to 8 and enable gradient accumulation with 4 steps to maintain effective batch size.. Reducing the batch size to 8 directly lowers GPU memory usage per forward pass, and enabling gradient accumulation with 4 steps allows the model to simulate the original effective batch size of 32 (8 × 4 = 32) without increasing memory footprint. This approach resolves the CUDA out-of-memory error while keeping the same instance type (ml.p3.2xlarge) and without incurring additional costs.

Answer

Enable SageMaker Managed Spot Training to reduce costs and use the savings to upgrade to a ml.p3.8xlarge instance.

Answer

Switch to a CPU-based instance like ml.c5.2xlarge to avoid GPU memory constraints.

Answer

Reduce the maximum sequence length to 128 tokens to lower memory consumption.

Question 10

A team is deploying a TensorFlow model on a SageMaker real-time endpoint with automatic scaling. They set the scaling policy to target an average CPU utilization of 50%. However, during traffic spikes, the endpoint experiences high latency and 503 errors. The instance type is ml.c5.large. What should the team do to resolve this while minimizing cost?

Accepted Answer

Add a scaling policy based on the number of concurrent requests per instance. Scaling based on CPU utilization alone is often insufficient for inference workloads where latency is the primary concern. By adding a scaling policy based on the number of concurrent requests per instance, the team can proactively scale out before CPU saturation occurs, reducing latency and eliminating 503 errors. SageMaker's automatic scaling supports multiple target tracking metrics, and using concurrent requests per instance aligns more closely with the actual demand on the model serving container.

Answer

Pre-warm the endpoint by keeping a fixed number of additional instances

Answer

Increase the scale-in cooldown period to avoid frequent downsizing

Answer

Change the instance type to a larger one like ml.c5.xlarge to handle the spikes