MLA-C01 ML Solution Monitoring, Maintenance, and Security Practice Test 7 — 15 Questions

Question 1

A company has deployed a machine learning model on Amazon SageMaker and wants to automatically detect when the distribution of input features deviates significantly from the training data distribution. Which SageMaker feature should they use?

Accepted Answer

SageMaker Model Monitor – Data Quality Monitoring. SageMaker Model Monitor – Data Quality Monitoring is the correct choice because it is specifically designed to detect deviations in the distribution of input features compared to the training data distribution. It continuously monitors incoming inference requests and compares statistical properties (e.g., mean, variance, or histogram) against a baseline computed from the training dataset, alerting when drift is detected.

Answer

SageMaker Clarify

Answer

SageMaker Edge Manager

Answer

SageMaker Model Monitor – Model Quality Monitoring

Question 2

A machine learning engineer notices that the latency of a SageMaker endpoint has increased over time. They need to identify which component (model inference vs. pre/post-processing) contributes most to the latency. Which CloudWatch metrics should they examine?

Accepted Answer

ModelLatency and OverheadLatency. SageMaker endpoints emit CloudWatch metrics that break down total latency into model inference time (ModelLatency) and the time spent in pre/post-processing (OverheadLatency). By comparing these two metrics, the engineer can pinpoint whether the bottleneck is in the inference code or in the custom preprocessing/postprocessing logic. Option D directly provides both metrics needed for this root-cause analysis.

Answer

Latency and ModelLatency

Answer

Invocations and 4XXError

Answer

5XXError and MemoryUtilization

Question 3

A financial services company must deploy a SageMaker endpoint that only accepts traffic from within a VPC and encrypts all data at rest and in transit using customer-managed KMS keys. They also need to prevent inter-container traffic from being visible to other users. Which combination of settings fulfills these requirements?

Accepted Answer

Configure the endpoint with VPC-only mode, enable inter-container traffic encryption, and specify a KMS key for endpoint data encryption. VPC-only mode restricts traffic to the VPC. Inter-container traffic encryption ensures data in transit between containers is encrypted. KMS key specified in the endpoint configuration encrypts data at rest.

Answer

Deploy the endpoint in a private subnet and use SageMaker Model Monitor to detect unauthorized access

Answer

Attach a security group that only allows inbound traffic from the VPC CIDR and enable data encryption using a KMS key

Answer

Enable network isolation mode and use a VPC configuration with no public internet access

Question 4

A data science team wants to automate the retraining of a model whenever SageMaker Model Monitor detects a significant drift in data quality. They need the least amount of custom code. Which approach should they use?

Accepted Answer

Set up a CloudWatch Alarm on the Model Monitor violation metric that sends a notification to an SNS topic, which triggers an AWS Lambda function to start a SageMaker pipeline. CloudWatch Alarms can be set on Model Monitor violation metrics. The alarm triggers an SNS topic that invokes a Lambda function to start a retraining pipeline. This requires minimal custom code.

Answer

Write a custom script on an EC2 instance that polls the Model Monitor endpoint every hour and triggers retraining if drift is detected

Answer

Configure an Amazon EventBridge rule that listens for Model Monitor violation events and directly triggers a SageMaker Pipeline execution

Answer

Use SageMaker Autopilot to automatically retrain the model when performance degrades

Question 5

A company wants to track the lineage of their ML models, including the training dataset, hyperparameters, and training job used to produce each model version. Which AWS service should they use?

Accepted Answer

SageMaker ML Lineage Tracking. SageMaker ML Lineage Tracking is the correct choice because it is purpose-built to record and query the provenance of ML models, capturing relationships between datasets, training jobs, hyperparameters, and model versions. It creates a directed acyclic graph (DAG) of entities (e.g., artifacts, actions, contexts) that allows you to trace how a specific model version was produced, which directly meets the requirement for lineage tracking.

Answer

Amazon DynamoDB

Answer

AWS Glue Data Catalog

Answer

Amazon S3 object tagging

Question 6

A company uses SageMaker Model Monitor to detect bias drift in their real-time inference endpoint. They have collected ground truth labels and want to monitor for bias across different demographic groups. Which type of monitoring should they configure?

Accepted Answer

SageMaker Clarify – Bias Drift Monitoring. SageMaker Clarify offers post-deployment bias monitoring that uses ground truth labels to compute bias metrics (e.g., difference in positive outcome rates) over time.

Answer

SageMaker Model Monitor – Feature Attribution Drift Monitoring

Answer

SageMaker Model Monitor – Model Quality Monitoring

Answer

SageMaker Model Monitor – Data Quality Monitoring

Question 7

A company deploys a model with SageMaker and wants to monitor for concept drift. They have noticed that the relationship between input features and the target variable has changed, causing model accuracy to degrade. However, the input data distribution remains stable. Which type of drift is this, and what is the most appropriate response strategy?

Accepted Answer

Concept drift; retrain the model with newly collected labeled data. This is concept drift because the relationship between input features and the target variable has changed while the input data distribution remains stable. The most appropriate response is to retrain the model with newly collected labeled data that reflects the current relationship, as concept drift requires updating the model's learned mapping from features to labels.

Answer

Concept drift; ignore the change as long as input distribution remains stable

Answer

Data drift; update the baseline statistics and continue monitoring

Answer

Data drift; retrain the model with the latest training data

Question 8

A machine learning engineer wants to reduce costs for a SageMaker real-time endpoint that experiences predictable traffic patterns with low traffic at night and high traffic during business hours. Which approach is most cost-effective while maintaining availability?

Accepted Answer

Configure an auto-scaling policy with target tracking based on the Invocations metric. Target tracking scaling policies automatically adjust instance count based on a metric like invocation count or CPU utilization. Step scaling can also be used but target tracking is simpler and more cost-effective for predictable patterns.

Answer

Use a single large instance type that can handle peak traffic at all times

Answer

Set up a scheduled scaling policy that manually adjusts instances at fixed times

Answer

Use SageMaker Inference Recommender to find the optimal instance type and then manually set instance count

Question 9

A company needs to ensure that their SageMaker Studio environment is only accessible from within their corporate network and that all data processed in Studio remains encrypted. Which configuration should they use?

Accepted Answer

Enable SageMaker Studio in VPC-only mode and use a KMS key for data encryption. Option C is correct because enabling SageMaker Studio in VPC-only mode ensures that the Studio environment is accessible only from within the corporate network by routing all traffic through a VPC with no public internet access. Additionally, using a KMS key for data encryption provides customer-managed encryption for data at rest and in transit within the Studio environment, meeting the encryption requirement.

Answer

Use SageMaker Studio with public internet access and enable AWS WAF

Answer

Place SageMaker Studio in a public subnet and use security groups to restrict access

Answer

Use IAM policies to allow only corporate IP addresses and enable encryption at rest with an S3 bucket key

Question 10

A company wants to deploy a foundation model from SageMaker JumpStart with the lowest possible inference cost, given that latency requirements are flexible. They have a mix of traffic volumes. Which approach should they take?

Accepted Answer

Select the smallest instance type that meets throughput requirements and enable automatic scaling. SageMaker JumpStart provides pre-built models; for cost optimization, choosing the smallest suitable instance type and enabling auto-scaling based on demand reduces cost while handling varying traffic.

Answer

Use SageMaker Savings Plans to get a discount on on-demand instances

Answer

Deploy the model on the largest GPU instance to handle peak load

Answer

Deploy the model on a serverless inference endpoint

Question 11

A company uses SageMaker Model Monitor's feature attribution drift monitoring with SHAP. They receive an alert that the average SHAP value for a particular feature has increased significantly compared to the baseline. The feature's input distribution has not changed. What does this likely indicate?

Accepted Answer

Concept drift in the model. SHAP values measure the contribution of each feature to the model's predictions. A change in SHAP values while input distribution is stable suggests the model has learned a new reliance on that feature, i.e., the relationship between the feature and the target has changed — concept drift.

Answer

The feature is no longer relevant to predictions

Answer

A bug in the SHAP computation

Answer

Data drift in that feature

Question 12

A company wants to allow a SageMaker model in one AWS account to be accessed by a different AWS account for inference. They need to maintain security and compliance. Which approach meets the requirement?

Accepted Answer

Attach a resource-based policy to the SageMaker endpoint that grants the other account's IAM role invoke permissions. Cross-account access can be achieved by using resource-based policies on the SageMaker model or endpoint, combined with appropriate IAM roles in the consuming account.

Answer

Use AWS PrivateLink to expose the SageMaker endpoint privately and grant access via security groups

Answer

Create an IAM role in the source account and share the role ARN with the target account

Answer

Share the model artifacts via an S3 bucket with cross-account bucket policies and let the other account deploy independently

Question 13

A company has a SageMaker real-time endpoint that serves predictions. They want to set up automated monitoring and remediation for when the number of 5XX errors exceeds a threshold. Which TWO steps should they take? (Choose TWO.)

Accepted Answer

Configure the CloudWatch Alarm to publish to an SNS topic. Option B is correct because a CloudWatch Alarm on the 5XXError metric can be configured to publish to an SNS topic, enabling automated notifications or remediation actions (e.g., via Lambda) when the alarm state is triggered. This is the standard AWS approach for alerting on endpoint errors without custom polling.

Answer

Use SageMaker Model Monitor to detect 5XX errors

Answer

Set up a scheduled EventBridge rule to check 5XXError every minute

Answer

Write a custom script on EC2 to poll the endpoint and check for errors

Question 14

A company wants to track the lineage of their ML models for reproducibility and auditability. Which THREE services or features should they use together to achieve this? (Choose THREE.)

Accepted Answer

Amazon S3 versioning. Amazon S3 versioning is correct because it preserves every version of an object stored in an S3 bucket, including model artifacts, datasets, and configuration files. By enabling versioning, you can retrieve and revert to any previous version of a model artifact, which is essential for reproducibility and auditability. This directly supports tracking the lineage of ML models by ensuring that the exact input data and model binaries used in a specific experiment are never overwritten or lost.

Answer

AWS CloudTrail

Answer

AWS Config

Question 15

A company is deploying a SageMaker endpoint and must meet strict security requirements: no public internet access, all inter-container traffic must be encrypted, and all data at rest must be encrypted with a customer-managed KMS key. Which THREE configurations should they apply? (Choose THREE.)

Accepted Answer

Configure the endpoint to use VPC-only mode

Answer

Attach a security group that only allows inbound traffic from the VPC CIDR

Answer

Enable network isolation mode on the endpoint