MLA-C01 ML Solution Monitoring, Maintenance, and Security Practice Test 4 — 15 Questions

Question 1

A company wants to reduce costs for a real-time inference endpoint that experiences predictable traffic spikes during business hours and low traffic at night. Which auto-scaling policy is MOST cost-effective while maintaining performance?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Scheduled scaling that increases instances before business hours and decreases after. Scheduled scaling directly aligns capacity with the predictable traffic pattern (business hours vs. night), allowing you to proactively add instances before demand increases and remove them afterward. This avoids the cost of over-provisioning during low-traffic periods and the latency of reactive scaling, making it the most cost-effective approach for a known, recurring schedule.

Answer

Step scaling based on CPU utilization

Answer

Manual scaling by the operations team

Answer

Target tracking with a custom metric for response time

Question 2

A team trains a model using SageMaker and wants to ensure that the training job cannot access the internet, but needs to access a private S3 bucket in the same VPC. Which configuration should they use?

Accepted Answer

Enable network isolation and provide VPC subnet and security group. SageMaker training jobs can be configured with VPC settings. Enabling network isolation prevents internet access, but the job still needs VPC connectivity to reach S3 via VPC endpoints. Setting the VPC config allows access to private resources.

Answer

Enable network isolation only, without VPC configuration

Answer

Disable network isolation but restrict security group rules

Answer

Use a public S3 bucket with bucket policies

Question 3

A data scientist deploys a model and wants to monitor the endpoint's invocation latency. They notice that the CloudWatch metric 'ModelLatency' is high, but 'OverheadLatency' is low. Which statement correctly interprets these metrics?

Accepted Answer

The model inference time is the bottleneck; consider optimizing the model or using a faster instance type. The 'ModelLatency' metric measures the time taken by the SageMaker model container to process a single request, including inference and any preprocessing/postprocessing within the container. 'OverheadLatency' measures the time spent on SageMaker infrastructure (e.g., network I/O, request queuing, and response handling). When ModelLatency is high and OverheadLatency is low, the bottleneck is clearly the model inference time itself, not the infrastructure overhead. Therefore, optimizing the model (e.g., quantization, pruning) or upgrading to a faster instance type (e.g., GPU vs. CPU) is the correct remediation.

Answer

The SageMaker overhead is causing the delay; check endpoint configuration

Answer

The endpoint is overloaded; increase the number of instances

Answer

The network latency is high; move the endpoint closer to clients

Question 4

An organization uses SageMaker Studio and needs to restrict Studio's internet access while allowing users to install custom packages from a private PyPI mirror hosted in a VPC. Which networking configuration should they use?

Accepted Answer

Disable internet access for Studio and configure VPC-only mode, then use a VPC endpoint to the private PyPI mirror. SageMaker Studio in VPC-only mode (no internet access) combined with VPC-only mode for the domain and VPC endpoints to private PyPI mirror enables package installation from the private mirror. NAT gateway or internet gateway would allow internet access, which is not desired.

Answer

Use a NAT gateway to allow outbound traffic to the private PyPI mirror

Answer

Disable internet access for Studio and rely on SageMaker's default VPC configuration

Answer

Enable internet access for Studio and use a VPC endpoint to the private PyPI mirror

Question 5

A machine learning team wants to monitor bias in a deployed model's predictions on an ongoing basis. Which AWS service should they use to schedule bias monitoring jobs and generate reports?

Accepted Answer

SageMaker Clarify with bias drift monitoring. SageMaker Clarify is the correct choice because it provides built-in bias drift monitoring capabilities that can be scheduled to run on a recurring basis. It evaluates predictions against pre-training and post-training bias metrics (e.g., DPL, DI, CDDL) and generates detailed reports, making it the only service designed specifically for ongoing bias monitoring in deployed models.

Answer

Amazon QuickSight with Athona queries

Answer

AWS CloudTrail for prediction logging

Answer

SageMaker Model Monitor with data quality monitoring

Question 6

A company uses SageMaker Inference Recommender to select the optimal endpoint configuration. After running the recommender, they receive a recommendation for a specific instance type and initial instance count. What should they do next to optimize costs over time?

Accepted Answer

Set up auto-scaling with a target tracking policy based on the recommended metric. SageMaker Inference Recommender provides a baseline configuration. To optimize costs, they should apply auto-scaling with a target tracking policy based on the recommended metric, such as invocation count or latency.

Answer

Use the recommended configuration without changes, as it is already optimal

Answer

Purchase a Savings Plan for the recommended instance type to reduce hourly cost

Answer

Manually adjust the instance count daily based on observed traffic

Question 7

A machine learning team wants to detect concept drift in a production model. Which TWO actions should they take? (Choose TWO)

Accepted Answer

Enable data capture on the endpoint to collect ground truth labels. Concept drift is detected by comparing model predictions to actual outcomes (ground truth). Capturing ground truth and using model quality monitoring is essential. Data quality monitoring would not detect concept drift.

Answer

Set up SageMaker Model Monitor data quality monitoring schedule

Answer

Use SageMaker Clarify for feature attribution drift

Answer

Enable daily retraining to automatically correct drift

Question 8

A data science team wants to automate the retraining of a model when data drift is detected. Which TWO AWS services should they use in combination to achieve this? (Choose TWO)

Accepted Answer

SageMaker Model Monitor. SageMaker Model Monitor detects data drift and can publish to CloudWatch Metrics. CloudWatch Alarms can trigger an SNS topic, which invokes a Lambda function that starts a SageMaker Pipeline for retraining.

Answer

AWS Cloud9

Answer

Amazon DynamoDB

Answer

Amazon Kinesis Data Analytics

Question 9

A machine learning engineer wants to monitor a deployed model for data drift. Which SageMaker feature should they use to automatically detect drift in the input data distribution compared to the training data baseline?

Accepted Answer

SageMaker Model Monitor. SageMaker Model Monitor can be configured to run monitoring jobs that compare live inference data against a baseline created from training data to detect data drift.

Answer

SageMaker Pipelines

Answer

SageMaker Debugger

Answer

SageMaker Clarify

Question 10

A machine learning team needs to ensure that all model training and inference jobs within SageMaker Studio run in a private network without internet access. The team also requires that inter-container traffic within the same training job be encrypted. Which configurations should they combine?

Accepted Answer

Enable network isolation mode and inter-container traffic encryption. VPC-only mode restricts all traffic to the VPC, and inter-container traffic encryption ensures data in transit within the job is encrypted. These two settings together meet the requirements.

Answer

Configure SageMaker Studio in VPC-only mode and use KMS encryption

Answer

Use a VPC with a NAT gateway and enable network isolation

Answer

Enable inter-container traffic encryption and use a VPC with VPC endpoints

Question 11

A company wants to reduce costs for a SageMaker real-time endpoint that has variable traffic. Which feature allows the endpoint to automatically adjust instance count based on demand?

Accepted Answer

Auto Scaling for SageMaker endpoints. Application Auto Scaling for SageMaker endpoints allows dynamic adjustment of instance count based on CloudWatch metrics such as CPU utilization or invocations per instance.

Answer

SageMaker Savings Plans

Answer

SageMaker Inference Recommender

Answer

SageMaker Model Monitor

Question 12

A hospital deploys a model to predict patient readmission risk. To comply with regulations, they must ensure that the model's predictions do not show bias against any demographic group over time. Which service should they use for ongoing monitoring?

Accepted Answer

SageMaker Clarify. SageMaker Clarify is the correct service because it is specifically designed to detect bias in ML model predictions and can be configured for ongoing monitoring. It provides bias metrics (e.g., difference in positive proportion, disparate impact) and can run on a schedule to continuously evaluate predictions against demographic groups, ensuring regulatory compliance over time.

Answer

AWS Audit Manager

Answer

SageMaker Model Monitor

Answer

Amazon Macie

Question 13

A team uses SageMaker ML Lineage Tracking to capture the metadata of their ML workflow. They want to query the lineage to see which model version was trained from a specific dataset. Which Lineage Tracking entity represents the dataset?

Accepted Answer

Artifact. In SageMaker ML Lineage Tracking, datasets are represented as Artifacts. Actions represent processes like training, and Contexts group related entities.

Answer

Association

Answer

Action

Answer

Context

Question 14

A security team requires that all data used by a SageMaker training job be encrypted at rest using a customer-managed KMS key. The data is stored in an S3 bucket that is already encrypted with SSE-KMS. What additional configuration is needed on the SageMaker training job?

Accepted Answer

Specify the KMS key as the VolumeKmsKeyId and OutputKmsKeyId in the training job configuration. When the input data is encrypted with a customer-managed KMS key, you must specify the same KMS key in the VolumeKmsKeyId parameter of the training job to encrypt the ML storage volume, and also set the OutputKmsKeyId for output encryption.

Answer

Enable inter-container traffic encryption

Answer

No additional configuration is required because S3 SSE-KMS automatically applies

Answer

Use network isolation mode

Question 15

An organization wants to schedule a retraining pipeline to run every Sunday night. Which AWS service should they use to trigger the pipeline on a schedule?

Accepted Answer

Amazon EventBridge. Amazon EventBridge is the correct choice because it provides a scheduled event source using cron or rate expressions to trigger target services at specified times, such as every Sunday night. It can directly invoke an AWS Step Functions state machine or a Lambda function to start the retraining pipeline, making it the native scheduling service for event-driven workflows in AWS.

Answer

AWS Lambda

Answer

AWS Step Functions

Answer

Amazon SQS