MLA-C01 ML Solution Monitoring, Maintenance, and Security — Timed 20 Questions

Question 1

A financial services company must deploy a SageMaker endpoint that processes sensitive customer data. They require that all traffic between the endpoint and the model containers be encrypted, and that the endpoint cannot be accessed from outside a specific VPC. Which combination of settings should they use?

20 minute time limit — choose an answer to begin.

Accepted Answer

Enable inter-container traffic encryption and configure the endpoint with VPC-only mode. Inter-container traffic encryption ensures that data between the SageMaker endpoint and the model containers is encrypted in transit, typically using TLS. Configuring the endpoint with VPC-only mode restricts all inference traffic to the specified VPC, preventing any access from outside that VPC. This combination directly addresses the requirements for encrypted inter-container traffic and VPC-restricted access.

Answer

Use a private VPC and enable data encryption at rest using KMS

Answer

Enable network isolation mode and inter-container traffic encryption

Answer

Deploy the endpoint in a private subnet and use a VPC endpoint for SageMaker API

Question 2

A machine learning engineer is setting up a retraining pipeline that triggers when concept drift is detected. They plan to use CloudWatch Alarms to monitor the model's accuracy metric. When drift is detected, they want to automatically start a SageMaker training job. Which architecture should they use?

Accepted Answer

CloudWatch Alarm → SNS → Lambda → SageMaker Training Job. CloudWatch Alarms cannot directly invoke SageMaker training jobs; they require an intermediary like SNS to trigger a Lambda function, which then calls the SageMaker API to start the training job. This pattern ensures reliable decoupling and allows the Lambda function to handle any preprocessing or conditional logic before launching the job.

Answer

CloudWatch Alarm → SQS → Lambda → SageMaker Training Job

Answer

CloudWatch Alarm → EventBridge → SageMaker Training Job

Answer

CloudWatch Alarm → Lambda directly (without SNS)

Question 3

A company needs to give a data science team in another AWS account access to deploy a model from a shared model registry. Which approach should they use to grant cross-account access?

Accepted Answer

Attach a resource-based policy to the model registry granting the other account access. Resource-based policies (also called resource policies) can be attached to the model registry to allow cross-account access. IAM roles are used for intra-account access, not for granting access to resources in another account.

Answer

Create an IAM role in the other account with permissions to access the registry

Answer

Use AWS Organizations SCP to allow cross-account access

Answer

Share the model artifacts via Amazon S3 bucket policy and use an IAM role

Question 4

A company wants to automatically trigger model retraining when SageMaker Model Monitor detects data drift. Which TWO services should they integrate to achieve this automation? (Choose two.)

Accepted Answer

AWS Lambda. Amazon EventBridge [CORRECT] is the right integration point because SageMaker Model Monitor publishes drift-detection events (e.g., 'SageMaker Model Monitoring' state changes and 'Monitoring Execution' status events) to the default EventBridge event bus, allowing rules to match those events and route them to targets for automation. AWS Lambda [CORRECT] is the correct compute target because an EventBridge rule can invoke a Lambda function that programmatically starts the retraining workflow (for example, calling CreateTrainingJob or starting a pipeline), providing the serverless automation the scenario requires. AWS Step Functions (A) is a valid orchestration service but is not the event source that Model Monitor emits to, so it is not one of the two services needed to detect drift and trigger the action. Amazon CloudWatch Alarms (C) can react to metrics but Model Monitor's drift events are delivered through EventBridge, not as CloudWatch alarms, so it is not the correct trigger mechanism here. Amazon SNS (E) can fan out notifications but does not itself detect drift or initiate retraining, so it does not satisfy the automation requirement.

Answer

AWS Step Functions

Answer

Amazon CloudWatch Alarms

Answer

Amazon SNS

Question 5

A company wants to secure data in transit between the client and SageMaker endpoint, and between containers in the same endpoint. Which THREE configurations should they apply? (Choose three.)

Accepted Answer

Enable network isolation for the model containers. Option B is correct because enabling network isolation for the model containers prevents the containers from making outbound network calls, which is a required security control when securing traffic between containers in the same endpoint. Option D is correct because configuring the endpoint to use HTTPS (TLS) encrypts data in transit between the client and the SageMaker endpoint. Option E is correct because enabling inter-container traffic encryption encrypts the traffic between containers within the same endpoint, directly addressing the second part of the requirement. Option A is incorrect because KMS encryption at rest protects stored data, not data in transit. Option C is incorrect because a VPC-only endpoint configuration controls network access and routing but does not by itself encrypt traffic between the client and endpoint or between containers.

Answer

Enable data encryption at rest using KMS

Answer

Use a VPC-only endpoint configuration

Question 6

A machine learning engineer is setting up model quality monitoring for a binary classification model. They have ground truth labels available in Amazon S3. Which TWO steps are required to configure model quality monitoring? (Choose two.)

Accepted Answer

Create a schedule for the monitoring job to run at regular intervals. Option D is correct because Amazon SageMaker Model Monitor requires a baseline that captures expected model quality metrics (such as accuracy, precision, recall, F1, and AUC) computed from training data and ground truth labels; without this baseline, the monitoring job has no reference to compare against. Option C is correct because model quality monitoring runs as a scheduled processing job, so you must create a monitoring schedule (for example, hourly or daily) that specifies the baseline, the endpoint or batch data to evaluate, and the ground truth S3 location. Option A is not required to configure monitoring itself; a CloudWatch alarm is an optional downstream action for alerting on emitted metrics. Option B is unrelated because SageMaker Clarify addresses bias detection and explainability, not model quality drift. Option E is not required for model quality monitoring because data capture is a prerequisite for data quality monitoring of endpoint traffic, whereas model quality monitoring compares predictions against ground truth labels that are already available in S3.

Answer

Create a CloudWatch Alarm for accuracy degradation

Answer

Use Amazon SageMaker Clarify for bias detection

Answer

Enable data capture on the endpoint to collect predictions

Question 7

A company deploys a model for credit risk assessment on a SageMaker endpoint. To comply with internal policies, they must ensure that the endpoint only allows inference requests from within a specific VPC and that the data is encrypted at rest. Which configuration meets these requirements?

Accepted Answer

Configure endpoint in VPC-only mode and enable KMS encryption for the endpoint. Configuring the SageMaker endpoint in VPC-only mode ensures that inference requests can only originate from within the specified VPC, satisfying the network restriction requirement. Enabling KMS encryption for the endpoint encrypts the model data and inference data at rest using a customer-managed key, meeting the encryption-at-rest policy.

Answer

Enable network isolation mode and use KMS for endpoint encryption

Answer

Configure endpoint in VPC-only mode and enable inter-container traffic encryption

Answer

Deploy endpoint in a public subnet and use a security group to restrict traffic

Question 8

A machine learning team uses SageMaker Pipelines and wants to automatically retrain a model when data drift is detected. They have set up Model Monitor to publish drift violations to CloudWatch. Which approach provides a COMPLETE serverless retraining pipeline triggered by drift detection?

Accepted Answer

Configure a CloudWatch Alarm on drift metric → SNS topic → Lambda function that starts the SageMaker Pipeline execution. The recommended pattern: CloudWatch Alarm triggers on drift metric → SNS message → Lambda function (receives SNS) → starts SageMaker Pipeline execution. EventBridge could also trigger on SNS events, but Lambda is simplest. EventBridge can schedule retraining but does not directly react to specific drift alarms. Step Functions would add unnecessary complexity.

Answer

Use SageMaker Model Monitor to directly invoke a SageMaker Pipeline when drift is detected

Answer

Use EventBridge to schedule retraining daily regardless of drift

Answer

Create an EventBridge rule that triggers on Model Monitor drift events to start the pipeline

Question 9

A company notices that the prediction distribution of their deployed model has shifted significantly from the training data distribution, but the input data distribution remains unchanged. Which type of drift is occurring, and what is the MOST likely cause?

Accepted Answer

Concept drift; the underlying relationship between features and target has changed. Concept drift occurs when the statistical relationship between the input features and the target variable changes over time, even if the input distribution itself remains stable. Here, the input data distribution is unchanged, but the prediction distribution has shifted, indicating that the model's learned mapping from features to target is no longer valid. This is the defining characteristic of concept drift, as opposed to data drift, which would involve a change in the input distribution.

Answer

Data drift; the training data is no longer representative of the current environment

Answer

Bias drift; the model is making unfair predictions against a protected group

Answer

Model drift; the model has degraded due to software issues

Question 10

An ML team wants to monitor the cost of their SageMaker endpoints. They have observed that some endpoints are underutilized. Which AWS offering can help them reduce costs by committing to a consistent amount of usage in exchange for a lower price?

Accepted Answer

SageMaker Savings Plans. SageMaker Savings Plans offer a flexible pricing model where you commit to a consistent amount of compute usage (measured in dollars per hour) over a 1- or 3-year term, in exchange for a lower price compared to On-Demand rates. This directly addresses the goal of reducing costs for underutilized endpoints by allowing the team to pay a discounted rate for the baseline usage they commit to, regardless of whether the endpoint is fully utilized.

Answer

SageMaker endpoint auto-scaling

Answer

SageMaker Managed Spot Training

Answer

SageMaker Inference Recommender

Question 11

A company deploys a model in a different AWS account for production. They want to allow the production account to invoke the model endpoint from a SageMaker notebook in the same account, while keeping the model in the original account. Which configuration is required?

Accepted Answer

Attach a resource policy to the SageMaker model in the model account that grants invoke permissions to the production account's IAM role. To allow a production account to invoke a SageMaker model endpoint in another account, you must attach a resource policy to the model in the model account. This policy grants invoke permissions to the production account's IAM role. The production account's role must also have identity-based permissions to invoke the endpoint. This cross-account access is enabled via resource policies on the model.

Answer

Create an IAM role in the production account with cross-account trust to assume a role in the model account

Answer

Use SageMaker Model Registry to share the model across accounts

Answer

Set up VPC peering between the two accounts and use private DNS

Question 12

A company uses SageMaker Model Monitor to detect data drift. They want to receive alerts when drift is detected and automatically trigger a retraining pipeline. Which TWO steps should they implement? (Select TWO.)

Accepted Answer

Set up an SNS subscription that triggers a Lambda function to start the SageMaker Pipeline. Amazon SNS can be used to publish a notification when Model Monitor detects data drift, and a Lambda function subscribed to that SNS topic can invoke the SageMaker Pipeline to trigger retraining. This decouples the monitoring from the pipeline execution, allowing for flexible, event-driven automation. Option D is correct because Model Monitor emits CloudWatch metrics for data quality violations, and you can create a CloudWatch Alarm on those metrics to publish to an SNS topic, which can then trigger a retraining pipeline via Lambda or other integrations.

Answer

Configure Model Monitor to directly invoke a SageMaker Pipeline when drift is detected

Answer

Configure a SageMaker Processing job to run periodically and check drift

Answer

Create an EventBridge rule that triggers on Model Monitor drift events to start the pipeline

Question 13

A financial services company uses SageMaker Studio. They require that all Studio traffic remains within the corporate network and that user notebooks cannot access the internet. Which TWO configurations should they implement? (Select TWO.)

Accepted Answer

Create the SageMaker Studio domain in a VPC with VPC endpoints for SageMaker and other services. Option D is correct because creating the SageMaker Studio domain inside a VPC and attaching interface VPC endpoints (AWS PrivateLink) for SageMaker and dependent services such as S3, ECR, and CloudWatch keeps all Studio traffic on the corporate network without traversing the public internet. Option E is correct because SageMaker Studio's VPC-only mode (network access type 'VPCOnly') removes direct internet access from the JupyterServer and KernelGateway apps, so user notebooks cannot reach the internet and must use VPC endpoints for any AWS calls. Option A is not right because security groups alone do not provide the required private connectivity to AWS services and are not the supported mechanism for enforcing Studio's no-internet behavior. Option B is incorrect because network isolation applies to training and processing jobs, not to interactive Studio notebook traffic. Option C is incorrect because KMS encryption protects data at rest and does not restrict network egress or internet access.

Answer

Use security groups to block outbound traffic from Studio notebooks

Answer

Enable network isolation mode for all training jobs

Answer

Enable KMS encryption for Studio data

Question 14

A financial services company deploys a fraud detection model with a SageMaker endpoint. They need to ensure that all data sent to the endpoint is encrypted in transit and at rest, and that the endpoint cannot be accessed from the public internet. Which combination of settings should they use?

Accepted Answer

Enable VPC-only mode, inter-container traffic encryption, and use a KMS key for endpoint encryption. Enabling VPC-only mode ensures the endpoint is not accessible from the public internet, inter-container traffic encryption secures data between containers in the endpoint, and a KMS key provides encryption at rest for the endpoint's storage. Together these settings satisfy the requirements for encryption in transit and at rest plus network isolation. This combination is the only one that addresses all three security controls.

Answer

Use endpoint data encryption with an AWS managed key and enable public endpoint access

Answer

Enable inter-container traffic encryption and disable VPC-only mode

Answer

Deploy the endpoint in a private subnet without SageMaker VPC-only mode

Question 15

A team wants to automatically retrain a model whenever data drift is detected on their SageMaker endpoint. Which AWS service should they use to invoke a retraining pipeline in response to a CloudWatch Alarm?

Accepted Answer

Amazon SNS topic triggering an AWS Lambda function. Amazon SNS can directly subscribe to a CloudWatch Alarm and, upon alarm state change, publish a message to an SNS topic. That topic can then trigger an AWS Lambda function, which invokes the retraining pipeline (e.g., SageMaker Processing or training job). This creates a fully managed, serverless event-driven workflow without needing custom polling or additional orchestration services.

Answer

AWS Step Functions directly from CloudWatch Alarm

Answer

SageMaker Processing job scheduled via EventBridge

Answer

Amazon SQS queue polling by a custom application

Question 16

A company uses SageMaker Model Monitor to track feature attribution drift with SHAP. They notice that the SHAP values have changed significantly for a feature, while the model performance remains stable. What is the MOST likely interpretation?

Accepted Answer

The model's behavior has changed, which may lead to future performance degradation and warrants investigation. A significant change in SHAP values indicates that the model's internal feature importance has shifted, even if overall performance metrics like accuracy or loss remain stable. This is a classic sign of concept drift or model behavior drift, where the model's decision boundary has changed for that feature, which can lead to future performance degradation as the drift accumulates. SageMaker Model Monitor tracks feature attribution drift separately from data drift, and a change in SHAP values without data drift suggests the model is relying on the feature differently, warranting investigation.

Answer

The ground truth labels are incorrect, causing the drift

Answer

Data drift has occurred for that feature

Answer

The model is learning equally from all features, so no action is needed

Question 17

A machine learning engineer deploys a multi-model endpoint using SageMaker. They need to track which model version was used for each inference request for compliance purposes. Which service should they integrate to capture this lineage?

Accepted Answer

SageMaker ML Lineage Tracking. SageMaker ML Lineage Tracking is the correct service because it is specifically designed to capture and query the lineage of machine learning artifacts, including model versions, datasets, and inference requests. By integrating with SageMaker endpoints, it automatically records the model version used for each inference, enabling compliance auditing without custom code.

Answer

AWS CloudTrail

Answer

SageMaker Model Monitor

Answer

Amazon DynamoDB with custom logging

Question 18

A team trains a model using SageMaker and wants to ensure that the training job cannot access the internet, but needs to access a private S3 bucket in the same VPC. Which configuration should they use?

Accepted Answer

Enable network isolation and provide VPC subnet and security group. Network isolation is the SageMaker training job setting that disables all outbound network access from the training container, including internet and other AWS services. However, to still reach a private S3 bucket in the same VPC, the job must be attached to a VPC subnet and security group so that traffic to the S3 VPC endpoint (or the bucket's private IP) is permitted. Combining network isolation with VPC configuration is the only way to block internet access while allowing private S3 access.

Answer

Enable network isolation only, without VPC configuration

Answer

Disable network isolation but restrict security group rules

Answer

Use a public S3 bucket with bucket policies

Question 19

A machine learning team wants to monitor bias in a deployed model's predictions on an ongoing basis. Which AWS service should they use to schedule bias monitoring jobs and generate reports?

Accepted Answer

SageMaker Clarify with bias drift monitoring. SageMaker Clarify is the correct choice because it provides built-in bias drift monitoring capabilities that can be scheduled to run on a recurring basis. It evaluates predictions against pre-training and post-training bias metrics (e.g., DPL, DI, CDDL) and generates detailed reports, making it the only service designed specifically for ongoing bias monitoring in deployed models.

Answer

Amazon QuickSight with Athona queries

Answer

AWS CloudTrail for prediction logging

Answer

SageMaker Model Monitor with data quality monitoring

Question 20

A machine learning team wants to detect concept drift in a production model. Which TWO actions should they take? (Choose TWO)

Accepted Answer

Enable data capture on the endpoint to collect ground truth labels. Option D is correct because enabling data capture on the endpoint records the actual inference requests and responses (input/output data) to Amazon S3, which is the prerequisite for obtaining the ground truth labels needed to measure whether the model's predictions are degrading over time. Option E is correct because a SageMaker Model Monitor model quality monitoring schedule compares captured predictions against ground truth labels to compute metrics like accuracy, precision, and recall, and it can emit CloudWatch alerts when those metrics drift — which is exactly how concept drift is detected. Option A is not correct because data quality monitoring only checks input feature statistics (e.g., missing values, distribution shifts in features) against a baseline, which detects data drift rather than concept drift in the model's predictive relationship. Option B is not correct because Clarify feature attribution drift (bias/explainability monitoring) measures changes in feature importance, not the degradation of model accuracy against ground truth. Option C is not correct because daily retraining is a remediation strategy, not a detection mechanism, and it does not by itself identify or quantify concept drift.

Answer

Set up SageMaker Model Monitor data quality monitoring schedule

Answer

Use SageMaker Clarify for feature attribution drift

Answer

Enable daily retraining to automatically correct drift