MLA-C01 ML Solution Monitoring, Maintenance, and Security Practice Test 4 — 15 Questions

Question 1

A machine learning engineer wants to monitor a deployed model for data drift. Which SageMaker feature should they use to automatically detect drift in the input data distribution compared to the training data baseline?

Accepted Answer

SageMaker Model Monitor. SageMaker Model Monitor can be configured to run monitoring jobs that compare live inference data against a baseline created from training data to detect data drift.

Answer

SageMaker Pipelines

Answer

SageMaker Debugger

Answer

SageMaker Clarify

Question 2

A team has deployed a real-time inference endpoint. They need to monitor the latency experienced by end users, including network overhead. Which CloudWatch metric should they use?

Accepted Answer

OverheadLatency. The OverheadLatency metric captures the total time from when the client sends a request to when it receives the response, including network round-trip time and any intermediate processing. This is the correct metric for monitoring end-user latency because it accounts for network overhead, unlike ModelLatency which only measures the time the model takes to generate a prediction inside the endpoint.

Answer

ModelLatency

Answer

Latency

Answer

Invocations

Question 3

A company wants to automatically trigger a retraining pipeline when concept drift is detected in their deployed model. Which combination of services should they use?

Accepted Answer

SageMaker Model Monitor → CloudWatch Alarm → SNS → Lambda. Option C is correct because SageMaker Model Monitor detects concept drift by analyzing model predictions against a baseline, then publishes metrics to CloudWatch. A CloudWatch Alarm triggers when drift exceeds a threshold, sending a notification via SNS to invoke a Lambda function, which starts the retraining pipeline. This end-to-end integration ensures automated, event-driven retraining without manual intervention.

Answer

SageMaker Model Monitor → Lambda

Answer

CloudWatch Events → SageMaker Training Job

Answer

SageMaker Clarify → SNS → Step Functions

Question 4

A machine learning team needs to ensure that all model training and inference jobs within SageMaker Studio run in a private network without internet access. The team also requires that inter-container traffic within the same training job be encrypted. Which configurations should they combine?

Accepted Answer

Enable network isolation mode and inter-container traffic encryption. VPC-only mode restricts all traffic to the VPC, and inter-container traffic encryption ensures data in transit within the job is encrypted. These two settings together meet the requirements.

Answer

Configure SageMaker Studio in VPC-only mode and use KMS encryption

Answer

Use a VPC with a NAT gateway and enable network isolation

Answer

Enable inter-container traffic encryption and use a VPC with VPC endpoints

Question 5

A data scientist uses SageMaker Model Monitor to track feature attribution drift. Which technique does SageMaker Model Monitor use to compute feature attributions?

Accepted Answer

SHAP. SageMaker Model Monitor uses SHAP (SHapley Additive exPlanations) to compute feature attributions for model explainability and drift detection. SHAP provides a unified measure of feature importance based on cooperative game theory, ensuring consistent and locally accurate attributions across all features.

Answer

Permutation Feature Importance

Answer

Partial Dependence Plots

Answer

LIME

Question 6

A company wants to reduce costs for a SageMaker real-time endpoint that has variable traffic. Which feature allows the endpoint to automatically adjust instance count based on demand?

Accepted Answer

Auto Scaling for SageMaker endpoints. Application Auto Scaling for SageMaker endpoints allows dynamic adjustment of instance count based on CloudWatch metrics such as CPU utilization or invocations per instance.

Answer

SageMaker Savings Plans

Answer

SageMaker Inference Recommender

Answer

SageMaker Model Monitor

Question 7

A hospital deploys a model to predict patient readmission risk. To comply with regulations, they must ensure that the model's predictions do not show bias against any demographic group over time. Which service should they use for ongoing monitoring?

Accepted Answer

SageMaker Clarify. SageMaker Clarify is the correct service because it is specifically designed to detect bias in ML model predictions and can be configured for ongoing monitoring. It provides bias metrics (e.g., difference in positive proportion, disparate impact) and can run on a schedule to continuously evaluate predictions against demographic groups, ensuring regulatory compliance over time.

Answer

AWS Audit Manager

Answer

SageMaker Model Monitor

Answer

Amazon Macie

Question 8

A team uses SageMaker ML Lineage Tracking to capture the metadata of their ML workflow. They want to query the lineage to see which model version was trained from a specific dataset. Which Lineage Tracking entity represents the dataset?

Accepted Answer

Artifact. In SageMaker ML Lineage Tracking, datasets are represented as Artifacts. Actions represent processes like training, and Contexts group related entities.

Answer

Association

Answer

Action

Answer

Context

Question 9

A security team requires that all data used by a SageMaker training job be encrypted at rest using a customer-managed KMS key. The data is stored in an S3 bucket that is already encrypted with SSE-KMS. What additional configuration is needed on the SageMaker training job?

Accepted Answer

Specify the KMS key as the VolumeKmsKeyId and OutputKmsKeyId in the training job configuration. When the input data is encrypted with a customer-managed KMS key, you must specify the same KMS key in the VolumeKmsKeyId parameter of the training job to encrypt the ML storage volume, and also set the OutputKmsKeyId for output encryption.

Answer

Enable inter-container traffic encryption

Answer

No additional configuration is required because S3 SSE-KMS automatically applies

Answer

Use network isolation mode

Question 10

An organization wants to schedule a retraining pipeline to run every Sunday night. Which AWS service should they use to trigger the pipeline on a schedule?

Accepted Answer

Amazon EventBridge. Amazon EventBridge is the correct choice because it provides a scheduled event source using cron or rate expressions to trigger target services at specified times, such as every Sunday night. It can directly invoke an AWS Step Functions state machine or a Lambda function to start the retraining pipeline, making it the native scheduling service for event-driven workflows in AWS.

Answer

AWS Lambda

Answer

AWS Step Functions

Answer

Amazon SQS

Question 11

A company uses SageMaker Studio and wants to restrict studio user access to only the VPC. They also need to encrypt the data exchanged between the Studio app and the kernel gateway. Which configuration should they apply?

Accepted Answer

Enable VPC-only mode and inter-container traffic encryption. Option C is correct because enabling VPC-only mode restricts all SageMaker Studio traffic to the VPC, preventing any internet-bound communication, while inter-container traffic encryption ensures that data exchanged between the Studio app and the kernel gateway is encrypted in transit using TLS. This combination satisfies both requirements: network confinement to the VPC and encryption of inter-component traffic.

Answer

Configure Studio to run in a private subnet and disable internet access

Answer

Use a VPC endpoint for SageMaker and enable KMS encryption

Answer

Enable network isolation and use a VPC with a NAT gateway

Question 12

A team monitors a production endpoint and notices a sudden increase in 5XXError count. Which of the following is the most likely cause?

Accepted Answer

The model container is out of memory or crashing. A sudden increase in 5XX errors, particularly HTTP 503 or 502, typically indicates that the model container is failing to process requests due to resource exhaustion (e.g., OOM kills) or a crash in the inference process. In a production ML endpoint, such errors often stem from the container running out of memory, leading to the container being terminated by the orchestrator (e.g., Kubernetes OOMKill) or the application crashing internally, which directly causes 5XX responses.

Answer

The endpoint is under-provisioned and requests are throttled

Answer

The input data format has changed

Answer

The model is returning predictions with high latency

Question 13

A machine learning team needs to monitor a deployed model for both data drift and concept drift. Which TWO approaches should they implement? (Select TWO.)

Accepted Answer

Set up SageMaker Model Monitor for data quality monitoring. SageMaker Model Monitor can be configured for data quality (data drift) and model quality (concept drift) monitoring. Data drift monitors input distribution changes, while model quality monitors prediction accuracy against ground truth.

Answer

Use SageMaker Clarify for bias monitoring

Answer

Configure CloudWatch Logs Insights to query inference logs

Answer

Enable SageMaker Debugger during inference

Question 14

A company wants to ensure that a SageMaker endpoint can only be invoked from within a specific VPC and that the data in transit is encrypted. Which THREE steps should they take? (Select THREE.)

Accepted Answer

Attach a resource policy to the endpoint that restricts access to the VPC endpoint. To restrict access to a VPC, use a VPC endpoint for SageMaker. To enforce that only requests from that VPC are accepted, use a resource policy that denies requests unless they come via the VPC endpoint. Inter-container traffic encryption secures data between containers, but for endpoint invocation, using a VPC endpoint with encryption ensures data in transit is encrypted.

Answer

Enable network isolation mode on the endpoint

Answer

Use a VPC with a NAT gateway and configure the endpoint to use the VPC

Answer

Enable inter-container traffic encryption on the endpoint

Question 15

A data science team uses SageMaker to train models. They need to track the lineage of each model, including the dataset used, training job, and hyperparameters. Which TWO SageMaker features can they use together? (Select TWO.)

Accepted Answer

SageMaker SDK. SageMaker ML Lineage Tracking captures the relationships between artifacts, actions, and contexts. SageMaker SDK automates lineage tracking when using the SDK for training jobs. SageMaker Experiments can also be used to track runs, but lineage tracking is specifically for provenance.

Answer

SageMaker Model Registry

Answer

SageMaker Pipelines

Answer

SageMaker Experiments