MLA-C01 ML Solution Monitoring, Maintenance, and Security Practice Test 9 — 15 Questions

Question 1

A machine learning engineer notices that the latency of a SageMaker endpoint has increased over time. They need to identify which component (model inference vs. pre/post-processing) contributes most to the latency. Which CloudWatch metrics should they examine?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

ModelLatency and OverheadLatency. SageMaker endpoints emit CloudWatch metrics that break down total latency into model inference time (ModelLatency) and the time spent in pre/post-processing (OverheadLatency). By comparing these two metrics, the engineer can pinpoint whether the bottleneck is in the inference code or in the custom preprocessing/postprocessing logic. Option D directly provides both metrics needed for this root-cause analysis.

Answer

Latency and ModelLatency

Answer

Invocations and 4XXError

Answer

5XXError and MemoryUtilization

Question 2

A company wants to track the lineage of their ML models, including the training dataset, hyperparameters, and training job used to produce each model version. Which AWS service should they use?

Accepted Answer

SageMaker ML Lineage Tracking. SageMaker ML Lineage Tracking is the correct choice because it is purpose-built to record and query the provenance of ML models, capturing relationships between datasets, training jobs, hyperparameters, and model versions. It creates a directed acyclic graph (DAG) of entities (e.g., artifacts, actions, contexts) that allows you to trace how a specific model version was produced, which directly meets the requirement for lineage tracking.

Answer

Amazon DynamoDB

Answer

AWS Glue Data Catalog

Answer

Amazon S3 object tagging

Question 3

A company uses SageMaker Model Monitor to detect bias drift in their real-time inference endpoint. They have collected ground truth labels and want to monitor for bias across different demographic groups. Which type of monitoring should they configure?

Accepted Answer

SageMaker Clarify – Bias Drift Monitoring. SageMaker Clarify – Bias Drift Monitoring is the correct choice because it specifically monitors bias metrics over time by comparing ground truth labels with model predictions across different demographic groups. It detects changes in bias metrics such as disparate impact, which is exactly what the company needs.

Answer

SageMaker Model Monitor – Feature Attribution Drift Monitoring

Answer

SageMaker Model Monitor – Model Quality Monitoring

Answer

SageMaker Model Monitor – Data Quality Monitoring

Question 4

A company deploys a model with SageMaker and wants to monitor for concept drift. They have noticed that the relationship between input features and the target variable has changed, causing model accuracy to degrade. However, the input data distribution remains stable. Which type of drift is this, and what is the most appropriate response strategy?

Accepted Answer

Concept drift; retrain the model with newly collected labeled data. This is concept drift because the relationship between input features and the target variable has changed while the input data distribution remains stable. The most appropriate response is to retrain the model with newly collected labeled data that reflects the current relationship, as concept drift requires updating the model's learned mapping from features to labels.

Answer

Concept drift; ignore the change as long as input distribution remains stable

Answer

Data drift; update the baseline statistics and continue monitoring

Answer

Data drift; retrain the model with the latest training data

Question 5

A company wants to deploy a foundation model from SageMaker JumpStart with the lowest possible inference cost, given that latency requirements are flexible. They have a mix of traffic volumes. Which approach should they take?

Accepted Answer

Select the smallest instance type that meets throughput requirements and enable automatic scaling. SageMaker JumpStart provides pre-built models; for cost optimization, choosing the smallest suitable instance type and enabling auto-scaling based on demand reduces cost while handling varying traffic.

Answer

Use SageMaker Savings Plans to get a discount on on-demand instances

Answer

Deploy the model on the largest GPU instance to handle peak load

Answer

Deploy the model on a serverless inference endpoint

Question 6

A company uses SageMaker Model Monitor's feature attribution drift monitoring with SHAP. They receive an alert that the average SHAP value for a particular feature has increased significantly compared to the baseline. The feature's input distribution has not changed. What does this likely indicate?

Accepted Answer

Concept drift in the model. Feature attribution drift monitoring compares SHAP value distributions between baseline and current data. If the input distribution of a feature is unchanged but its average SHAP value shifts significantly, the model's learned relationship between that feature and the target has changed — this is the definition of concept drift. The model itself is unchanged, but the underlying mapping from inputs to outputs in the real world has shifted, so the same input values now contribute differently to predictions.

Answer

The feature is no longer relevant to predictions

Answer

A bug in the SHAP computation

Answer

Data drift in that feature

Question 7

A company has a SageMaker real-time endpoint that serves predictions. They want to set up automated monitoring and remediation for when the number of 5XX errors exceeds a threshold. Which TWO steps should they take? (Choose TWO.)

Accepted Answer

Configure the CloudWatch Alarm to publish to an SNS topic. A CloudWatch Alarm on the 5XXError metric can be configured to publish to an SNS topic, enabling automated notifications or remediation actions (e.g., via Lambda) when the alarm state is triggered. This is the standard AWS approach for alerting on endpoint errors without custom polling.

Answer

Use SageMaker Model Monitor to detect 5XX errors

Answer

Set up a scheduled EventBridge rule to check 5XXError every minute

Answer

Write a custom script on EC2 to poll the endpoint and check for errors

Question 8

A company wants to track the lineage of their ML models for reproducibility and auditability. Which THREE services or features should they use together to achieve this? (Choose THREE.)

Accepted Answer

Amazon S3 versioning. Amazon S3 versioning (A) is correct because it preserves every version of the datasets and model artifacts stored in S3, so a given training run can be tied to the exact immutable object version used, which is essential for reproducibility and auditability. SageMaker Experiments (B) is correct because it records experiment runs, trial components, parameters, metrics, and input/output artifacts, giving the structured record of each training attempt needed to reproduce results. SageMaker ML Lineage Tracking (D) is correct because it automatically creates and stores entities and relationships (trials, trial components, artifacts, contexts, actions) forming a queryable lineage graph from data through training to the deployed model. AWS CloudTrail (C) only logs API activity and control-plane events for auditing who did what, not the data/model lineage relationships, so it does not by itself provide reproducibility lineage. AWS Config (E) evaluates and records resource configuration compliance over time, which is unrelated to tracking ML artifact provenance and experiment history.

Answer

AWS CloudTrail

Answer

AWS Config

Question 9

A financial services company is deploying a fraud detection model on SageMaker. To comply with regulations, they must ensure that the model's predictions are not biased against protected groups. They plan to monitor bias drift post-deployment using SageMaker Clarify. Which data inputs are required to configure Clarify's bias drift monitoring?

Accepted Answer

Baseline training data with ground truth labels and inference data with predictions. SageMaker Clarify's bias drift monitoring requires a baseline—specifically, the training data with ground truth labels—to establish the original bias metrics, and the inference data with predictions to compute post-deployment bias metrics. By comparing these two datasets, Clarify detects statistically significant shifts in bias over time, which is essential for regulatory compliance in fraud detection models.

Answer

Only the inference data with predictions

Answer

Only the ground truth labels for recent predictions

Answer

Only the training data with feature attributions

Question 10

A data science team uses SageMaker to train and deploy models. They need to track model lineage, including datasets, training jobs, and model versions, to ensure reproducibility. Which THREE actions should they take? (Select THREE)

Accepted Answer

Enable SageMaker ML Lineage Tracking. A is correct because SageMaker ML Lineage Tracking automatically captures the relationships between datasets, training jobs, and model versions, creating a directed acyclic graph (DAG) of the ML workflow. This enables full reproducibility by allowing you to trace which data and code produced a specific model, without manual intervention.

Answer

Store trained models in a public S3 bucket

Answer

Use SageMaker Experiments to organize training runs

Question 11

A team wants to secure SageMaker endpoints for a healthcare application. They must ensure data is encrypted at rest and in transit, and that the endpoint can only be accessed from within a VPC. Which THREE steps should they take? (Select THREE)

Accepted Answer

Use AWS KMS to encrypt the model artifacts and endpoint data

Answer

Store encryption keys in a public S3 bucket

Answer

Set the endpoint to use network isolation mode

Question 12

A company is deploying a foundation model using SageMaker JumpStart. They want to minimize inference costs while maintaining low latency. Which TWO strategies should they consider? (Select TWO)

Accepted Answer

Enable auto-scaling with a target tracking policy based on Invocations per instance. Auto-scaling with a target tracking policy based on Invocations per instance dynamically adjusts the number of instances to match demand, ensuring you only pay for the compute capacity you need while maintaining low latency. This avoids over-provisioning and reduces idle costs, directly addressing the goal of minimizing inference costs.

Answer

Enable data capture for all requests to analyze usage patterns

Answer

Use SageMaker Savings Plans for discounted compute rates

Answer

Deploy the model on a single large instance to maximize throughput

Question 13

A machine learning engineer wants to set up a retraining pipeline that triggers when model quality degrades. Which TWO components are essential for this automated retraining pipeline? (Select TWO)

Accepted Answer

CloudWatch Alarm on model quality metric

Answer

SageMaker Ground Truth to collect new labels

Answer

SageMaker Data Wrangler to preprocess data

Answer

EventBridge rule to schedule retraining weekly

Question 14

A company uses SageMaker Model Monitor for feature attribution drift monitoring with SHAP. Which THREE prerequisites must be in place before starting the monitoring schedule? (Select THREE)

Accepted Answer

A SageMaker Clarify processing job that computes SHAP values on the captured data. SageMaker Model Monitor requires a Clarify processing job to compute SHAP values on the captured data as part of the feature attribution drift monitoring setup. This job generates the necessary SHAP explainability values that are compared against the baseline to detect drift.

Answer

Ground truth labels for the inference data

Answer

Baseline constraints file for data quality

Question 15

A company wants to enable cross-account access to a SageMaker model endpoint. The model is in Account A, and Account B needs to invoke it. Which TWO steps are required? (Select TWO)

Accepted Answer

Attach a resource-based policy to the SageMaker model in Account A allowing access from Account B's IAM role. SageMaker endpoints support resource-based policies that allow cross-account access. By attaching a resource-based policy to the model endpoint in Account A, you can grant the IAM role from Account B explicit permission to invoke the endpoint. This is the standard AWS mechanism for cross-account SageMaker endpoint invocation without needing to duplicate the model.

Answer

Export the model from Account A and re-deploy in Account B

Answer

Configure VPC peering between the two accounts

Answer

Use a SageMaker notebook instance cross-account sharing