MLA-C01 ML Solution Monitoring, Maintenance, and Security Practice Test 2 — 15 Questions

Question 1

A company uses SageMaker Model Monitor to track feature attribution drift with SHAP. They notice that the SHAP values have changed significantly for a feature, while the model performance remains stable. What is the MOST likely interpretation?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The model's behavior has changed, which may lead to future performance degradation and warrants investigation. A significant change in SHAP values indicates that the model's internal feature importance has shifted, even if overall performance metrics like accuracy or loss remain stable. This is a classic sign of concept drift or model behavior drift, where the model's decision boundary has changed for that feature, which can lead to future performance degradation as the drift accumulates. SageMaker Model Monitor tracks feature attribution drift separately from data drift, and a change in SHAP values without data drift suggests the model is relying on the feature differently, warranting investigation.

Answer

The ground truth labels are incorrect, causing the drift

Answer

Data drift has occurred for that feature

Answer

The model is learning equally from all features, so no action is needed

Question 2

A machine learning engineer deploys a multi-model endpoint using SageMaker. They need to track which model version was used for each inference request for compliance purposes. Which service should they integrate to capture this lineage?

Accepted Answer

SageMaker ML Lineage Tracking. SageMaker ML Lineage Tracking is the correct service because it is specifically designed to capture and query the lineage of machine learning artifacts, including model versions, datasets, and inference requests. By integrating with SageMaker endpoints, it automatically records the model version used for each inference, enabling compliance auditing without custom code.

Answer

AWS CloudTrail

Answer

SageMaker Model Monitor

Answer

Amazon DynamoDB with custom logging

Question 3

A team trains a model using SageMaker and wants to ensure that the training job cannot access the internet, but needs to access a private S3 bucket in the same VPC. Which configuration should they use?

Accepted Answer

Enable network isolation and provide VPC subnet and security group. Network isolation is the SageMaker training job setting that disables all outbound network access from the training container, including internet and other AWS services. However, to still reach a private S3 bucket in the same VPC, the job must be attached to a VPC subnet and security group so that traffic to the S3 VPC endpoint (or the bucket's private IP) is permitted. Combining network isolation with VPC configuration is the only way to block internet access while allowing private S3 access.

Answer

Enable network isolation only, without VPC configuration

Answer

Disable network isolation but restrict security group rules

Answer

Use a public S3 bucket with bucket policies

Question 4

A machine learning team wants to monitor bias in a deployed model's predictions on an ongoing basis. Which AWS service should they use to schedule bias monitoring jobs and generate reports?

Accepted Answer

SageMaker Clarify with bias drift monitoring. SageMaker Clarify is the correct choice because it provides built-in bias drift monitoring capabilities that can be scheduled to run on a recurring basis. It evaluates predictions against pre-training and post-training bias metrics (e.g., DPL, DI, CDDL) and generates detailed reports, making it the only service designed specifically for ongoing bias monitoring in deployed models.

Answer

Amazon QuickSight with Athona queries

Answer

AWS CloudTrail for prediction logging

Answer

SageMaker Model Monitor with data quality monitoring

Question 5

A machine learning team wants to detect concept drift in a production model. Which TWO actions should they take? (Choose TWO)

Accepted Answer

Enable data capture on the endpoint to collect ground truth labels. Option D is correct because enabling data capture on the endpoint records the actual inference requests and responses (input/output data) to Amazon S3, which is the prerequisite for obtaining the ground truth labels needed to measure whether the model's predictions are degrading over time. Option E is correct because a SageMaker Model Monitor model quality monitoring schedule compares captured predictions against ground truth labels to compute metrics like accuracy, precision, and recall, and it can emit CloudWatch alerts when those metrics drift — which is exactly how concept drift is detected. Option A is not correct because data quality monitoring only checks input feature statistics (e.g., missing values, distribution shifts in features) against a baseline, which detects data drift rather than concept drift in the model's predictive relationship. Option B is not correct because Clarify feature attribution drift (bias/explainability monitoring) measures changes in feature importance, not the degradation of model accuracy against ground truth. Option C is not correct because daily retraining is a remediation strategy, not a detection mechanism, and it does not by itself identify or quantify concept drift.

Answer

Set up SageMaker Model Monitor data quality monitoring schedule

Answer

Use SageMaker Clarify for feature attribution drift

Answer

Enable daily retraining to automatically correct drift

Question 6

A company needs to secure a SageMaker real-time endpoint such that only authorized applications within a VPC can invoke the model, and all data in transit is encrypted. Which TWO configuration steps should they implement? (Choose TWO)

Accepted Answer

Enable VPC-only mode for the endpoint. Option D is correct because enabling VPC-only mode on a SageMaker endpoint restricts all inference traffic to the customer's VPC, ensuring only authorized applications inside that VPC can invoke the model and blocking any public internet access. Option E is correct because enabling inter-container traffic encryption ensures that data in transit between containers on the endpoint instances is encrypted, satisfying the requirement that all data in transit be protected. Option A is wrong because assigning a public IP to endpoint instances would expose the endpoint to the internet, directly contradicting the VPC-only access requirement. Option B is wrong because disabling API gateway logs reduces auditability and does not contribute to restricting access or encrypting traffic. Option C is wrong because a KMS key encrypts data at rest, not data in transit, so it does not meet the stated in-transit encryption requirement.

Answer

Assign a public IP to the endpoint instances

Answer

Disable SageMaker's API gateway logs

Answer

Use a KMS key for endpoint data encryption (at rest)

Question 7

A team has deployed a real-time inference endpoint. They need to monitor the latency experienced by end users, including network overhead. Which CloudWatch metric should they use?

Accepted Answer

OverheadLatency. OverheadLatency measures the additional time added by SageMaker infrastructure, including request handling, pre-processing, post-processing, and internal network overhead. It does not include model inference time (captured by ModelLatency) or client-side network latency. For monitoring end-user experience within the AWS environment, OverheadLatency is the metric that accounts for network overhead inside the service. If total end-to-end latency including client-side network is required, custom client-side metrics should be used.

Answer

ModelLatency

Answer

Latency

Answer

Invocations

Question 8

A security team requires that all data used by a SageMaker training job be encrypted at rest using a customer-managed KMS key. The data is stored in an S3 bucket that is already encrypted with SSE-KMS. What additional configuration is needed on the SageMaker training job?

Accepted Answer

Specify the KMS key as the VolumeKmsKeyId and OutputKmsKeyId in the training job configuration. To ensure data used by a SageMaker training job is encrypted at rest with a customer-managed KMS key, you must specify that key in the training job configuration. The VolumeKmsKeyId encrypts the attached EBS volumes, and the OutputKmsKeyId encrypts the output data in S3. This is necessary even if the input S3 bucket is already encrypted with SSE-KMS, because SageMaker needs to decrypt and re-encrypt data for processing.

Answer

Enable inter-container traffic encryption

Answer

No additional configuration is required because S3 SSE-KMS automatically applies

Answer

Use network isolation mode

Question 9

A company uses SageMaker Studio and wants to restrict studio user access to only the VPC. They also need to encrypt the data exchanged between the Studio app and the kernel gateway. Which configuration should they apply?

Accepted Answer

Enable VPC-only mode and inter-container traffic encryption. Enabling VPC-only mode restricts all SageMaker Studio traffic to the VPC, preventing any internet-bound communication, while inter-container traffic encryption ensures that data exchanged between the Studio app and the kernel gateway is encrypted in transit using TLS. This combination satisfies both requirements: network confinement to the VPC and encryption of inter-component traffic.

Answer

Configure Studio to run in a private subnet and disable internet access

Answer

Use a VPC endpoint for SageMaker and enable KMS encryption

Answer

Enable network isolation and use a VPC with a NAT gateway

Question 10

Which TWO steps should they take? (Select TWO.)

Accepted Answer

Attach a resource policy to the endpoint that restricts access to the VPC endpoint. To restrict access to a specific VPC, create a VPC endpoint for SageMaker (com.amazonaws.region.sagemaker.api) and attach a resource policy that denies requests unless they originate from that VPC endpoint. Options B, D, and E do not achieve this goal: network isolation mode prevents training jobs from accessing the internet but does not restrict endpoint invocation; a NAT gateway is used for outbound traffic, not inbound; inter-container traffic encryption secures communication between containers, not client-to-endpoint communication.

Answer

Enable network isolation mode on the endpoint

Answer

Use a VPC with a NAT gateway and configure the endpoint to use the VPC

Answer

Enable inter-container traffic encryption on the endpoint

Question 11

A data science team uses SageMaker to train models. They need to track the lineage of each model, including the dataset used, training job, and hyperparameters. Which TWO SageMaker features can they use together? (Select TWO.)

Accepted Answer

SageMaker SDK. SageMaker ML Lineage Tracking (C) is the correct service for this scenario because it automatically captures and stores lineage entities such as datasets, training jobs, model artifacts, and their relationships, allowing the team to trace the full history of a model including hyperparameters and the data used. The SageMaker SDK (A) is also correct because it provides the APIs and helper classes (for example, the LineageQuery and lineage context utilities) needed to programmatically create, query, and visualize that lineage information from notebooks or code. Together, the SDK lets the team interact with the lineage data that ML Lineage Tracking records. SageMaker Model Registry (B) only catalogs model versions and approval status, not the full training lineage. SageMaker Pipelines (D) orchestrates ML workflows but does not itself provide the lineage graph. SageMaker Experiments (E) tracks experiment runs and metrics, not the end-to-end lineage of datasets, jobs, and artifacts.

Answer

SageMaker Model Registry

Answer

SageMaker Pipelines

Answer

SageMaker Experiments

Question 12

A data science team uses SageMaker Studio with a VPC-only mode. They need to access a private S3 bucket in the same VPC to read training data. The SageMaker Studio domain is configured with VPC-only mode. Which configuration ensures the Studio notebook can access the S3 bucket without traversing the public internet?

Accepted Answer

Create an S3 Gateway Endpoint in the VPC and attach a bucket policy that allows access from the VPC. An S3 Gateway Endpoint provides a private, VPC-only route to S3 without traversing the public internet. By attaching a bucket policy that restricts access to the VPC endpoint, the SageMaker Studio domain (configured with VPC-only mode) can securely read training data from the private S3 bucket using AWS PrivateLink.

Answer

Set the S3 bucket as public and restrict access by source IP

Answer

Configure a NAT Gateway in the public subnet and route Studio traffic through it

Answer

Use a SageMaker execution role with a policy that allows s3:GetObject from any network

Question 13

A financial services company needs to deploy a SageMaker endpoint that only accepts inference requests from within a specific VPC and denies all public traffic. The endpoint must also encrypt data in transit between containers. How should the endpoint be configured?

Accepted Answer

Configure the endpoint with network isolation mode and enable inter-container traffic encryption. Enabling network isolation mode ensures the SageMaker endpoint is deployed within a VPC and cannot be accessed from the public internet, satisfying the requirement to deny all public traffic. Additionally, enabling inter-container traffic encryption (using TLS) encrypts data in transit between the containers hosting the model, meeting the encryption requirement. This configuration is specific to SageMaker endpoints and directly addresses both constraints.

Answer

Deploy the endpoint in a public subnet and restrict security group ingress to the VPC CIDR

Answer

Enable VPC-only mode for the endpoint and disable public access

Answer

Use a privateLink endpoint and enable data encryption at rest

Question 14

A company uses a SageMaker endpoint for real-time inference. To reduce costs, they want to implement auto-scaling based on the number of invocations per instance. They also need to ensure that scaling actions are recorded for audit. Which TWO steps should they take? (Select TWO.)

Accepted Answer

Create a target tracking scaling policy based on the InvocationsPerInstance metric. Option C is correct because before Application Auto Scaling can manage a SageMaker endpoint, the endpoint's variant must be registered as a scalable target using the register-scalable-target command, specifying the service namespace sagemaker, the resource ID (endpoint//variant/), and the min/max capacity. Option A is correct because a target tracking scaling policy using the predefined InvocationsPerInstance metric is exactly the mechanism that scales the endpoint based on invocations per instance, adjusting capacity to keep that metric at the target value. Option B is not required because Application Auto Scaling API calls are already recorded in AWS CloudTrail by default as management events, so no additional enablement step is needed for audit. Option D is wrong because SageMaker Model Monitor detects data/model drift and quality issues, not invocation counts for scaling. Option E is wrong because publishing custom CloudWatch metrics is unnecessary when the built-in InvocationsPerInstance metric already exists for target tracking.

Answer

Enable AWS CloudTrail to log auto-scaling API calls

Answer

Enable SageMaker Model Monitor to track invocation counts

Answer

Publish custom metrics to CloudWatch for endpoint utilization

Question 15

A company needs to give a data science team from another AWS account access to deploy models to a SageMaker endpoint in the company's account. The company wants to minimize administrative overhead while ensuring security. Which TWO steps should the company take? (Select TWO.)

Accepted Answer

Create a new SageMaker execution role in the company account and grant the external account's IAM role permission to pass it. Creating a SageMaker execution role in the company account and granting the external account's IAM role permission to pass it is a standard and secure method for cross-account access to SageMaker resources. Option A is incorrect because SageMaker models do not support resource-based policies for cross-account access; such policies are only available for SageMaker notebook instances. Options C, D, and E are also incorrect: sharing credentials or providing root user credentials violates security best practices, and a VPN is unnecessary for this use case.

Answer

Attach a resource-based policy to the SageMaker model granting access to the external account

Answer

Create an IAM role in the company account and share the credentials with the external team

Answer

Provide the external team with the company's AWS account root user credentials

Answer

Set up a VPN connection between the two accounts for network access