MLA-C01 ML Solution Monitoring, Maintenance, and Security Practice Test 2 — 15 Questions

Question 1

A data science team uses Amazon SageMaker Model Monitor to detect data drift in production. They notice that the schema of incoming data (number of features) has changed compared to the training baseline. Which type of monitor is BEST suited to detect this issue?

Accepted Answer

Data quality monitor. The Data quality monitor in SageMaker Model Monitor is specifically designed to detect violations in the input data schema, such as changes in the number of features, feature types, or missing values, by comparing incoming data against a baseline computed from the training dataset. Since the issue is a structural change in the schema (number of features), the Data quality monitor is the correct choice.

Answer

Bias drift monitor

Answer

Feature attribution drift monitor

Answer

Model quality monitor

Question 2

An ML engineer wants to be notified when the average inference latency of a SageMaker endpoint exceeds 500 ms for 2 consecutive evaluation periods. Which AWS service combination should they use?

Accepted Answer

CloudWatch Alarm + SNS. CloudWatch collects the 'Latency' metric from SageMaker endpoints. A CloudWatch Alarm can be set on this metric with a threshold of 500 ms for 2 consecutive periods. SNS then sends notification (email, SMS, etc.). Lambda is for automated remediation, not required for notification only.

Answer

SageMaker Model Monitor + SNS

Answer

EventBridge + Lambda

Answer

SageMaker Clarify + SNS

Question 3

A company deploys a model for credit risk assessment on a SageMaker endpoint. To comply with internal policies, they must ensure that the endpoint only allows inference requests from within a specific VPC and that the data is encrypted at rest. Which configuration meets these requirements?

Accepted Answer

Configure endpoint in VPC-only mode and enable KMS encryption for the endpoint. Option D is correct because configuring the SageMaker endpoint in VPC-only mode ensures that inference requests can only originate from within the specified VPC, satisfying the network restriction requirement. Enabling KMS encryption for the endpoint encrypts the model data and inference data at rest using a customer-managed key, meeting the encryption-at-rest policy.

Answer

Enable network isolation mode and use KMS for endpoint encryption

Answer

Configure endpoint in VPC-only mode and enable inter-container traffic encryption

Answer

Deploy endpoint in a public subnet and use a security group to restrict traffic

Question 4

A machine learning team uses SageMaker Pipelines and wants to automatically retrain a model when data drift is detected. They have set up Model Monitor to publish drift violations to CloudWatch. Which approach provides a COMPLETE serverless retraining pipeline triggered by drift detection?

Accepted Answer

Configure a CloudWatch Alarm on drift metric → SNS topic → Lambda function that starts the SageMaker Pipeline execution. The recommended pattern: CloudWatch Alarm triggers on drift metric → SNS message → Lambda function (receives SNS) → starts SageMaker Pipeline execution. EventBridge could also trigger on SNS events, but Lambda is simplest. EventBridge can schedule retraining but does not directly react to specific drift alarms. Step Functions would add unnecessary complexity.

Answer

Use SageMaker Model Monitor to directly invoke a SageMaker Pipeline when drift is detected

Answer

Use EventBridge to schedule retraining daily regardless of drift

Answer

Create an EventBridge rule that triggers on Model Monitor drift events to start the pipeline

Question 5

A financial institution uses SageMaker to train and deploy models. They need to track every experiment, model version, and deployment step for audit purposes. Which SageMaker feature should they use to capture the full lineage of artifacts, actions, and contexts?

Accepted Answer

SageMaker ML Lineage Tracking. SageMaker ML Lineage Tracking creates a graph of artifacts (datasets, models), actions (training jobs, deployment), and contexts (experiments). It provides a complete audit trail. Experiments alone track trials but not lineage. Model Registry tracks model versions but not full pipeline lineage. Clarify is for bias monitoring.

Answer

SageMaker Clarify

Answer

SageMaker Model Registry

Answer

SageMaker Experiments

Question 6

A company notices that the prediction distribution of their deployed model has shifted significantly from the training data distribution, but the input data distribution remains unchanged. Which type of drift is occurring, and what is the MOST likely cause?

Accepted Answer

Concept drift; the underlying relationship between features and target has changed. Concept drift occurs when the relationship between input features and target variable changes, causing prediction distribution shift. Data drift would be input distribution change. The scenario specifically says input distribution unchanged but predictions shifted, indicating concept drift. Possible cause: change in customer behavior or market conditions.

Answer

Data drift; the training data is no longer representative of the current environment

Answer

Bias drift; the model is making unfair predictions against a protected group

Answer

Model drift; the model has degraded due to software issues

Question 7

An ML team wants to monitor the cost of their SageMaker endpoints. They have observed that some endpoints are underutilized. Which AWS offering can help them reduce costs by committing to a consistent amount of usage in exchange for a lower price?

Accepted Answer

SageMaker Savings Plans. SageMaker Savings Plans offer a flexible pricing model where you commit to a consistent amount of compute usage (measured in dollars per hour) over a 1- or 3-year term, in exchange for a lower price compared to On-Demand rates. This directly addresses the goal of reducing costs for underutilized endpoints by allowing the team to pay a discounted rate for the baseline usage they commit to, regardless of whether the endpoint is fully utilized.

Answer

SageMaker endpoint auto-scaling

Answer

SageMaker Managed Spot Training

Answer

SageMaker Inference Recommender

Question 8

A company deploys a model in a different AWS account for production. They want to allow the production account to invoke the model endpoint from a SageMaker notebook in the same account, while keeping the model in the original account. Which configuration is required?

Accepted Answer

Attach a resource policy to the SageMaker model in the model account that grants invoke permissions to the production account's IAM role. Cross-account model access requires a resource policy on the model in Account A that grants invoke permissions to Account B. The production account's execution role must also have permission to invoke the model. SageMaker Model Registry does not handle cross-account inference. VPC peering is not sufficient for IAM permissions. IAM role cross-account trust is needed but the model resource policy is also necessary.

Answer

Create an IAM role in the production account with cross-account trust to assume a role in the model account

Answer

Use SageMaker Model Registry to share the model across accounts

Answer

Set up VPC peering between the two accounts and use private DNS

Question 9

A team wants to use SageMaker Clarify to monitor bias in their production model predictions. They have configured a bias drift monitor. What does SageMaker Clarify compare to detect bias drift?

Accepted Answer

Current bias metrics against a baseline bias metrics computed from training data. SageMaker Clarify bias drift monitor compares the bias metrics computed on current predictions against the baseline bias metrics computed from the training data or from an earlier period. It does not compare against model quality metrics or SHAP values. The baseline is typically established during the initial monitoring setup.

Answer

Current input data distribution against the training data distribution

Answer

Current SHAP feature attributions against baseline SHAP values

Answer

Current predictions against ground truth labels collected in real-time

Question 10

A machine learning engineer wants to deploy a pre-trained foundation model for text summarization using SageMaker JumpStart. Which of the following is a primary cost consideration when deploying such a model?

Accepted Answer

The cost of GPU instances required for low-latency inference. Foundation models are large and require GPU instances, which are more expensive. Inference cost is driven by instance type (GPU vs CPU) and the number of instances. While throughput and latency are performance considerations, the primary cost factor is the compute instance type. Data transfer costs are secondary. Fine-tuning costs are separate.

Answer

The cost of fine-tuning the model on custom data

Answer

The cost of data transfer for inference requests

Answer

The cost of storing the model artifacts in S3

Question 11

A company uses SageMaker endpoints for real-time inference. They want to automatically scale the number of instances based on the number of outstanding requests. Which auto-scaling policy type should they choose?

Accepted Answer

Target tracking scaling. Target tracking scaling adjusts the instance count to maintain a target metric value (e.g., average invocation count per instance). Step scaling uses predefined scaling adjustments based on alarm breaches but does not directly track a target. Simple scaling is not recommended for production. Scheduled scaling is for predictable patterns, not dynamic.

Answer

Scheduled scaling

Answer

Step scaling

Answer

Simple scaling

Question 12

An organization needs to ensure that all data transmitted between containers in a SageMaker training job is encrypted. In the training job configuration, which setting should they enable?

Accepted Answer

Enable inter-container traffic encryption. Option C is correct because SageMaker training jobs support inter-container traffic encryption, which ensures that data transmitted between containers (e.g., distributed training workers) is encrypted in transit. This setting uses TLS to protect the communication channel, meeting the organization's requirement for encrypted data transmission between containers.

Answer

Use a KMS key for data encryption

Answer

Configure the training job in VPC-only mode

Answer

Enable network isolation mode

Question 13

A company uses SageMaker Model Monitor to detect data drift. They want to receive alerts when drift is detected and automatically trigger a retraining pipeline. Which TWO steps should they implement? (Select TWO.)

Accepted Answer

Set up an SNS subscription that triggers a Lambda function to start the SageMaker Pipeline. Option C is correct because Amazon SNS can be used to publish a notification when Model Monitor detects data drift, and a Lambda function subscribed to that SNS topic can invoke the SageMaker Pipeline to trigger retraining. This decouples the monitoring from the pipeline execution, allowing for flexible, event-driven automation. Option D is correct because Model Monitor emits CloudWatch metrics for data quality violations, and you can create a CloudWatch Alarm on those metrics to publish to an SNS topic, which can then trigger a retraining pipeline via Lambda or other integrations.

Answer

Configure Model Monitor to directly invoke a SageMaker Pipeline when drift is detected

Answer

Configure a SageMaker Processing job to run periodically and check drift

Answer

Create an EventBridge rule that triggers on Model Monitor drift events to start the pipeline

Question 14

A financial services company uses SageMaker Studio. They require that all Studio traffic remains within the corporate network and that user notebooks cannot access the internet. Which TWO configurations should they implement? (Select TWO.)

Accepted Answer

Create the SageMaker Studio domain in a VPC with VPC endpoints for SageMaker and other services. To keep all traffic within the corporate network, SageMaker Studio must be configured with a VPC-only mode (no public internet access). Additionally, creating the Studio domain with a VPC and optionally using VPC endpoints for Studio and other services ensures traffic stays within AWS network. Network isolation mode applies to training jobs, not Studio. Security groups control traffic but do not prevent internet access. KMS encryption is for data at rest, not network.

Answer

Use security groups to block outbound traffic from Studio notebooks

Answer

Enable network isolation mode for all training jobs

Answer

Enable KMS encryption for Studio data

Question 15

An ML team uses SageMaker to deploy a model for real-time inference. They want to monitor and improve cost efficiency. Which THREE actions should they take? (Select THREE.)

Accepted Answer

Use SageMaker Inference Recommender to find the optimal instance type and count. SageMaker Inference Recommender runs load tests against your model to generate instance type and count recommendations that balance performance and cost. By selecting the optimal configuration, you avoid over-provisioned instances that waste money or under-provisioned ones that degrade user experience, directly improving cost efficiency.

Answer

Create a CloudWatch dashboard to monitor endpoint latency

Answer

Use SageMaker Managed Spot Training for endpoint instances