MLA-C01 ML Solution Monitoring, Maintenance, and Security Practice Test 5 — 15 Questions

Question 1

An ML engineer monitors a SageMaker endpoint for data drift. They set up SageMaker Model Monitor to compare inference data against a baseline created from the training dataset. The monitoring schedule runs daily and reports violations. Which monitoring type should be configured to detect if the distribution of a numerical feature in real-time inference data differs significantly from the training distribution?

Accepted Answer

Data quality monitoring. SageMaker Model Monitor's data quality monitoring detects feature distribution drift (statistical drift) between baseline and live data. Model quality monitoring requires ground truth labels, bias drift monitors fairness metrics, and feature attribution drift monitors SHAP values.

Answer

Feature attribution drift monitoring

Answer

Bias drift monitoring

Answer

Model quality monitoring

Question 2

A company wants to automate remediation when a SageMaker endpoint's latency exceeds a threshold for more than 5 minutes. The team needs to be notified and a Lambda function should be invoked to scale up the endpoint. Which combination of services should be used?

Accepted Answer

CloudWatch Alarm → SNS topic → Lambda function. CloudWatch Alarms monitor metrics and trigger actions. SNS can notify and invoke Lambda for automated remediation. EventBridge is used for scheduled events, not direct alarm actions.

Answer

EventBridge rule to trigger Lambda

Answer

CloudWatch Logs subscription filter → Lambda function

Answer

SageMaker Model Monitor → Lambda function

Question 3

A machine learning team trains a model in SageMaker and wants to track every step — from dataset version to hyperparameters to final model artifact — for reproducibility and audit compliance. Which SageMaker feature should they use?

Accepted Answer

SageMaker ML Lineage Tracking. SageMaker ML Lineage Tracking is the correct choice because it is specifically designed to create a directed acyclic graph (DAG) of every step in the ML workflow, including dataset versions, hyperparameters, training jobs, and model artifacts. This enables full reproducibility and audit compliance by capturing the provenance of each entity and their relationships, which is exactly what the question requires.

Answer

SageMaker Feature Store

Answer

SageMaker Experiments

Answer

SageMaker Model Registry

Question 4

A data science team uses SageMaker Studio with a VPC-only mode. They need to access a private S3 bucket in the same VPC to read training data. The SageMaker Studio domain is configured with VPC-only mode. Which configuration ensures the Studio notebook can access the S3 bucket without traversing the public internet?

Accepted Answer

Create an S3 Gateway Endpoint in the VPC and attach a bucket policy that allows access from the VPC. Option D is correct because an S3 Gateway Endpoint provides a private, VPC-only route to S3 without traversing the public internet. By attaching a bucket policy that restricts access to the VPC endpoint, the SageMaker Studio domain (configured with VPC-only mode) can securely read training data from the private S3 bucket using AWS PrivateLink.

Answer

Set the S3 bucket as public and restrict access by source IP

Answer

Configure a NAT Gateway in the public subnet and route Studio traffic through it

Answer

Use a SageMaker execution role with a policy that allows s3:GetObject from any network

Question 5

A company deploys a model for fraud detection. They want to monitor if the model's predictions become less accurate over time due to changes in the underlying data distribution, but they do not have immediate access to ground truth labels. Which type of drift should they monitor as a proxy?

Accepted Answer

Data drift. Data drift (option C) is the correct proxy to monitor when ground truth labels are unavailable because it detects changes in the input feature distribution over time. If the underlying data distribution shifts, the model's predictions are likely to become less accurate even if the relationship between features and labels remains stable. This allows teams to trigger retraining or investigation before model quality degrades.

Answer

Feature attribution drift

Answer

Model quality drift

Answer

Concept drift

Question 6

An organization needs to ensure that all data used for inference on a SageMaker endpoint is encrypted at rest. The endpoint uses a SageMaker-provided container. Which configuration should be applied?

Accepted Answer

Specify a KMS key in the endpoint configuration. SageMaker endpoints use AWS KMS for encryption at rest. By specifying a KMS key in the endpoint configuration, the data in the attached ML storage volume is encrypted. Inter-container traffic encryption is for encryption in transit.

Answer

Use a custom container with built-in encryption

Answer

Enable network isolation mode

Answer

Enable inter-container traffic encryption

Question 7

A company deploys a real-time inference endpoint with auto-scaling using a target tracking policy based on average Invocations per instance. They notice that during a traffic spike, the endpoint scales out too late, causing increased latency. They want to scale proactively before the spike. Which strategy should they implement?

Accepted Answer

Use a scheduled scaling action to add capacity before the expected spike. Scheduled scaling with Application Auto Scaling can anticipate traffic spikes. Pre-warming endpoints or using provisioned concurrency are not native SageMaker features. Step scaling reacts to deviations but still after the fact.

Answer

Enable provisioned concurrency on the endpoint

Answer

Pre-warm the endpoint by sending dummy requests

Answer

Switch to a step scaling policy with a higher cooldown period

Question 8

A team uses SageMaker Clarify to monitor bias drift on a deployed model. They have defined a baseline with training data and set up a monitoring schedule. After one month, they receive a violation report indicating that the post-training metrics have deviated from the baseline. What does this violation indicate?

Accepted Answer

The model's predictions relative to sensitive attributes have shifted compared to the training baseline. SageMaker Clarify bias drift monitoring compares predicted outcomes (post-training) against the baseline to detect changes in fairness metrics like disparate impact. It does not measure prediction accuracy or data quality.

Answer

The SHAP values for features have changed

Answer

The model's predictions are becoming less accurate

Answer

The distribution of input features has changed

Question 9

An ML engineer needs to monitor the operational health of a SageMaker endpoint, specifically the time taken for the container to process an inference request and the overhead added by SageMaker. Which two CloudWatch metrics should they examine?

Accepted Answer

ModelLatency and OverheadLatency. ModelLatency is the time taken by the model to respond, and OverheadLatency is the additional time added by SageMaker infrastructure. Invocations is count, not duration; Latency is total latency (ModelLatency + OverheadLatency).

Answer

ModelLatency and 4XXError

Answer

Latency and 5XXError

Answer

Invocations and Latency

Question 10

A company wants to implement a retraining pipeline that automatically triggers when SageMaker Model Monitor detects data drift. The retraining job should use the latest approved pipeline version in SageMaker Pipelines. Which approach meets these requirements?

Accepted Answer

Create an EventBridge rule that listens for SageMaker Model Monitor violation events and triggers a Lambda function that starts the pipeline. Option D is correct because it uses an EventBridge rule to listen for SageMaker Model Monitor violation events (e.g., `aws.sagemaker.model-monitoring-violation`), which then triggers a Lambda function that starts the latest approved pipeline version in SageMaker Pipelines. This creates an automated, event-driven retraining pipeline without manual intervention or scheduled polling.

Answer

Use a scheduled EventBridge rule to run the pipeline every day

Answer

Use SageMaker Model Monitor to update the model registry and trigger a deployment

Answer

Configure SageMaker Model Monitor to directly invoke a Lambda function on violation

Question 11

A financial services company needs to deploy a SageMaker endpoint that only accepts inference requests from within a specific VPC and denies all public traffic. The endpoint must also encrypt data in transit between containers. How should the endpoint be configured?

Accepted Answer

Configure the endpoint with network isolation mode and enable inter-container traffic encryption. Option D is correct because enabling network isolation mode ensures the SageMaker endpoint is deployed within a VPC and cannot be accessed from the public internet, satisfying the requirement to deny all public traffic. Additionally, enabling inter-container traffic encryption (using TLS) encrypts data in transit between the containers hosting the model, meeting the encryption requirement. This configuration is specific to SageMaker endpoints and directly addresses both constraints.

Answer

Deploy the endpoint in a public subnet and restrict security group ingress to the VPC CIDR

Answer

Enable VPC-only mode for the endpoint and disable public access

Answer

Use a privateLink endpoint and enable data encryption at rest

Question 12

A company plans to deploy a large foundation model using SageMaker JumpStart. They are concerned about costs because the model will be used intermittently. Which deployment option is MOST cost-effective for intermittent traffic?

Accepted Answer

Deploy as a serverless endpoint. Serverless endpoints in SageMaker automatically scale to zero when not in use, so you pay only for the compute time consumed during inference requests. This makes them the most cost-effective option for intermittent traffic, as you avoid paying for idle compute capacity.

Answer

Purchase SageMaker Savings Plans for the endpoint

Answer

Use a batch transform job for each request

Answer

Deploy as a real-time endpoint with a multi-model endpoint

Question 13

An ML team has deployed a model to a SageMaker real-time endpoint and wants to set up automated monitoring for model quality. Which TWO elements are required to configure SageMaker Model Monitor for model quality? (Select TWO.)

Accepted Answer

A ground truth labels dataset for comparison. Option C is correct because SageMaker Model Monitor for model quality requires a ground truth labels dataset to compare the model's predictions against actual outcomes. This comparison is essential for calculating quality metrics like accuracy, precision, recall, or F1 score, which indicate how well the model is performing over time.

Answer

SHAP values for feature attribution

Answer

A constraints file with allowed deviation thresholds

Answer

A baseline statistics file derived from the training data

Question 14

A company uses a SageMaker endpoint for real-time inference. To reduce costs, they want to implement auto-scaling based on the number of invocations per instance. They also need to ensure that scaling actions are recorded for audit. Which TWO steps should they take? (Select TWO.)

Accepted Answer

Create a target tracking scaling policy based on the InvocationsPerInstance metric. Option A is correct because the InvocationsPerInstance metric is a predefined SageMaker metric that Application Auto Scaling can use with a target tracking scaling policy. This policy automatically adjusts the number of instances to maintain a target value for invocations per instance, directly reducing costs by scaling down when demand is low.

Answer

Enable SageMaker Model Monitor to track invocation counts

Answer

Publish custom metrics to CloudWatch for endpoint utilization

Question 15

A company needs to give a data science team from another AWS account access to deploy models to a SageMaker endpoint in the company's account. The company wants to minimize administrative overhead while ensuring security. Which TWO steps should the company take? (Select TWO.)

Accepted Answer

Attach a resource-based policy to the SageMaker model granting access to the external account. Option A is correct because SageMaker models support resource-based policies that allow cross-account access. By attaching a resource-based policy to the SageMaker model, the company can grant the external AWS account's IAM role the necessary permissions to deploy the model to the endpoint without needing to manage additional IAM roles or share credentials. This approach minimizes administrative overhead by leveraging AWS's built-in cross-account access mechanism.

Answer

Create an IAM role in the company account and share the credentials with the external team

Answer

Provide the external team with the company's AWS account root user credentials

Answer

Set up a VPN connection between the two accounts for network access