MLA-C01 ML Solution Monitoring, Maintenance, and Security Practice Test 3 — 15 Questions

Question 1

An ML team wants to monitor the cost of their SageMaker endpoints. They have observed that some endpoints are underutilized. Which AWS offering can help them reduce costs by committing to a consistent amount of usage in exchange for a lower price?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

SageMaker Savings Plans. SageMaker Savings Plans offer a flexible pricing model where you commit to a consistent amount of compute usage (measured in dollars per hour) over a 1- or 3-year term, in exchange for a lower price compared to On-Demand rates. This directly addresses the goal of reducing costs for underutilized endpoints by allowing the team to pay a discounted rate for the baseline usage they commit to, regardless of whether the endpoint is fully utilized.

Answer

SageMaker endpoint auto-scaling

Answer

SageMaker Managed Spot Training

Answer

SageMaker Inference Recommender

Question 2

A company deploys a model in a different AWS account for production. They want to allow the production account to invoke the model endpoint from a SageMaker notebook in the same account, while keeping the model in the original account. Which configuration is required?

Accepted Answer

Attach a resource policy to the SageMaker model in the model account that grants invoke permissions to the production account's IAM role. Cross-account model access requires a resource policy on the model in Account A that grants invoke permissions to Account B. The production account's execution role must also have permission to invoke the model. SageMaker Model Registry does not handle cross-account inference. VPC peering is not sufficient for IAM permissions. IAM role cross-account trust is needed but the model resource policy is also necessary.

Answer

Create an IAM role in the production account with cross-account trust to assume a role in the model account

Answer

Use SageMaker Model Registry to share the model across accounts

Answer

Set up VPC peering between the two accounts and use private DNS

Question 3

A team wants to use SageMaker Clarify to monitor bias in their production model predictions. They have configured a bias drift monitor. What does SageMaker Clarify compare to detect bias drift?

Accepted Answer

Current bias metrics against a baseline bias metrics computed from training data. SageMaker Clarify bias drift monitor compares the bias metrics computed on current predictions against the baseline bias metrics computed from the training data or from an earlier period. It does not compare against model quality metrics or SHAP values. The baseline is typically established during the initial monitoring setup.

Answer

Current input data distribution against the training data distribution

Answer

Current SHAP feature attributions against baseline SHAP values

Answer

Current predictions against ground truth labels collected in real-time

Question 4

A machine learning engineer wants to deploy a pre-trained foundation model for text summarization using SageMaker JumpStart. Which of the following is a primary cost consideration when deploying such a model?

Accepted Answer

The cost of GPU instances required for low-latency inference. Foundation models are large and require GPU instances, which are more expensive. Inference cost is driven by instance type (GPU vs CPU) and the number of instances. While throughput and latency are performance considerations, the primary cost factor is the compute instance type. Data transfer costs are secondary. Fine-tuning costs are separate.

Answer

The cost of fine-tuning the model on custom data

Answer

The cost of data transfer for inference requests

Answer

The cost of storing the model artifacts in S3

Question 5

A company uses SageMaker endpoints for real-time inference. They want to automatically scale the number of instances based on the number of outstanding requests. Which auto-scaling policy type should they choose?

Accepted Answer

Target tracking scaling. Target tracking scaling adjusts the instance count to maintain a target metric value (e.g., average invocation count per instance). Step scaling uses predefined scaling adjustments based on alarm breaches but does not directly track a target. Simple scaling is not recommended for production. Scheduled scaling is for predictable patterns, not dynamic.

Answer

Scheduled scaling

Answer

Step scaling

Answer

Simple scaling

Question 6

An organization needs to ensure that all data transmitted between containers in a SageMaker training job is encrypted. In the training job configuration, which setting should they enable?

Accepted Answer

Enable inter-container traffic encryption. SageMaker training jobs support inter-container traffic encryption, which ensures that data transmitted between containers (e.g., distributed training workers) is encrypted in transit. This setting uses TLS to protect the communication channel, meeting the organization's requirement for encrypted data transmission between containers.

Answer

Use a KMS key for data encryption

Answer

Configure the training job in VPC-only mode

Answer

Enable network isolation mode

Question 7

A company uses SageMaker Model Monitor to detect data drift. They want to receive alerts when drift is detected and automatically trigger a retraining pipeline. Which TWO steps should they implement? (Select TWO.)

Accepted Answer

Set up an SNS subscription that triggers a Lambda function to start the SageMaker Pipeline. Amazon SNS can be used to publish a notification when Model Monitor detects data drift, and a Lambda function subscribed to that SNS topic can invoke the SageMaker Pipeline to trigger retraining. This decouples the monitoring from the pipeline execution, allowing for flexible, event-driven automation. Option D is correct because Model Monitor emits CloudWatch metrics for data quality violations, and you can create a CloudWatch Alarm on those metrics to publish to an SNS topic, which can then trigger a retraining pipeline via Lambda or other integrations.

Answer

Configure Model Monitor to directly invoke a SageMaker Pipeline when drift is detected

Answer

Configure a SageMaker Processing job to run periodically and check drift

Answer

Create an EventBridge rule that triggers on Model Monitor drift events to start the pipeline

Question 8

A financial services company uses SageMaker Studio. They require that all Studio traffic remains within the corporate network and that user notebooks cannot access the internet. Which TWO configurations should they implement? (Select TWO.)

Accepted Answer

Create the SageMaker Studio domain in a VPC with VPC endpoints for SageMaker and other services. To keep all traffic within the corporate network, SageMaker Studio must be configured with a VPC-only mode (no public internet access). Additionally, creating the Studio domain with a VPC and optionally using VPC endpoints for Studio and other services ensures traffic stays within AWS network. Network isolation mode applies to training jobs, not Studio. Security groups control traffic but do not prevent internet access. KMS encryption is for data at rest, not network.

Answer

Use security groups to block outbound traffic from Studio notebooks

Answer

Enable network isolation mode for all training jobs

Answer

Enable KMS encryption for Studio data

Question 9

An ML team uses SageMaker to deploy a model for real-time inference. They want to monitor and improve cost efficiency. Which THREE actions should they take? (Select THREE.)

Accepted Answer

Use SageMaker Inference Recommender to find the optimal instance type and count. SageMaker Inference Recommender runs load tests against your model to generate instance type and count recommendations that balance performance and cost. By selecting the optimal configuration, you avoid over-provisioned instances that waste money or under-provisioned ones that degrade user experience, directly improving cost efficiency.

Answer

Create a CloudWatch dashboard to monitor endpoint latency

Answer

Use SageMaker Managed Spot Training for endpoint instances

Question 10

A machine learning team deploys a model for loan approval. They want to monitor data drift on the real-time endpoint using SageMaker Model Monitor. Which set of actions should they take to set up data quality monitoring?

Accepted Answer

Enable data capture on the endpoint, generate a baseline from training data, create a data quality monitoring schedule, and set up a CloudWatch Alarm on violations. SageMaker Model Monitor requires a baseline from training data, then schedules monitoring jobs that compare live endpoint captures against that baseline. Alerts are sent via CloudWatch Alarms.

Answer

Use SageMaker Clarify to detect data drift on the endpoint

Answer

Create a model quality monitoring schedule directly on the endpoint without any baseline

Answer

Enable data capture and rely on SageMaker Model Monitor to automatically infer drift without a baseline

Question 11

A financial services company deploys a fraud detection model with a SageMaker endpoint. They need to ensure that all data sent to the endpoint is encrypted in transit and at rest, and that the endpoint cannot be accessed from the public internet. Which combination of settings should they use?

Accepted Answer

Enable VPC-only mode, inter-container traffic encryption, and use a KMS key for endpoint encryption. VPC-only mode ensures the endpoint is private. Inter-container traffic encryption is needed for data in transit between containers in multi-model endpoints. KMS encryption secures data at rest on the instance storage.

Answer

Use endpoint data encryption with an AWS managed key and enable public endpoint access

Answer

Enable inter-container traffic encryption and disable VPC-only mode

Answer

Deploy the endpoint in a private subnet without SageMaker VPC-only mode

Question 12

A data scientist notices that a production model's accuracy has degraded over the past week. The training data distribution remains unchanged, but the relationship between features and the target has shifted. Which type of drift is occurring, and which monitoring approach should be used?

Accepted Answer

Concept drift; use SageMaker Model Monitor model quality monitoring with ground truth labels. Concept drift occurs when the underlying relationship between features and target changes. Model quality monitoring (comparing predictions against ground truth) detects this. Data drift monitors feature distribution changes, which are not present here.

Answer

Bias drift; use SageMaker Clarify post-deployment bias monitoring

Answer

Data drift; use SageMaker Model Monitor data quality monitoring

Answer

Feature attribution drift; use SageMaker Clarify

Question 13

A team wants to automatically retrain a model whenever data drift is detected on their SageMaker endpoint. Which AWS service should they use to invoke a retraining pipeline in response to a CloudWatch Alarm?

Accepted Answer

Amazon SNS topic triggering an AWS Lambda function. Amazon SNS can directly subscribe to a CloudWatch Alarm and, upon alarm state change, publish a message to an SNS topic. That topic can then trigger an AWS Lambda function, which invokes the retraining pipeline (e.g., SageMaker Processing or training job). This creates a fully managed, serverless event-driven workflow without needing custom polling or additional orchestration services.

Answer

AWS Step Functions directly from CloudWatch Alarm

Answer

SageMaker Processing job scheduled via EventBridge

Answer

Amazon SQS queue polling by a custom application

Question 14

A company uses SageMaker Model Monitor to track feature attribution drift with SHAP. They notice that the SHAP values have changed significantly for a feature, while the model performance remains stable. What is the MOST likely interpretation?

Accepted Answer

The model's behavior has changed, which may lead to future performance degradation and warrants investigation. A significant change in SHAP values indicates that the model's internal feature importance has shifted, even if overall performance metrics like accuracy or loss remain stable. This is a classic sign of concept drift or model behavior drift, where the model's decision boundary has changed for that feature, which can lead to future performance degradation as the drift accumulates. SageMaker Model Monitor tracks feature attribution drift separately from data drift, and a change in SHAP values without data drift suggests the model is relying on the feature differently, warranting investigation.

Answer

The ground truth labels are incorrect, causing the drift

Answer

Data drift has occurred for that feature

Answer

The model is learning equally from all features, so no action is needed

Question 15

A machine learning engineer deploys a multi-model endpoint using SageMaker. They need to track which model version was used for each inference request for compliance purposes. Which service should they integrate to capture this lineage?

Accepted Answer

SageMaker ML Lineage Tracking. SageMaker ML Lineage Tracking is the correct service because it is specifically designed to capture and query the lineage of machine learning artifacts, including model versions, datasets, and inference requests. By integrating with SageMaker endpoints, it automatically records the model version used for each inference, enabling compliance auditing without custom code.

Answer

AWS CloudTrail

Answer

SageMaker Model Monitor

Answer

Amazon DynamoDB with custom logging