MLA-C01 ML Solution Monitoring, Maintenance and Security Practice Test 3 — 15 Questions

Question 1

An ML engineer is setting up monitoring for a SageMaker endpoint. Which THREE metrics should be monitored to detect performance issues? (Select THREE.)

Accepted Answer

Model latency. Model latency is a critical metric for detecting performance issues in a SageMaker endpoint because it directly measures the time taken to process inference requests. High latency can indicate resource bottlenecks, model inefficiency, or scaling problems, and it is essential for meeting service-level agreements (SLAs). Monitoring latency helps identify when the endpoint is underprovisioned or when the model itself has degraded in performance.

Answer

Invocations per second

Answer

DiskWriteBytes

Question 2

A company needs to secure a SageMaker notebook instance that contains sensitive data. Which THREE of the following are effective security measures? (Select THREE.)

Accepted Answer

Use IAM policies to restrict who can access the notebook instance.. Encrypting the EBS volume with KMS protects data at rest, IAM policies control access, and CloudTrail provides auditing. Disabling internet access is also good, but the question asks for three from the list.

Answer

Disable direct internet access and use a VPC with a NAT gateway for outbound.

Answer

Attach a lifecycle configuration that runs a script to download data from a public S3 bucket.

Question 3

Refer to the exhibit. A team has configured data capture for a SageMaker endpoint. The endpoint is returning predictions but no captured data appears in the S3 bucket. What is the most likely cause?

Accepted Answer

The IAM role for the endpoint does not have s3:PutObject permission.. The data capture configuration shows CurrentCaptureStatus as 'Running' and sampling percentage at 50%. Most likely the IAM role attached to the endpoint does not have s3:PutObject permission for the destination bucket.

Answer

The InitialSamplingPercentage is too low.

Answer

The capture status is 'Configured' but not 'Running'.

Answer

The endpoint is not receiving any traffic.

Question 4

Refer to the exhibit. A data scientist tries to deploy a model from an S3 bucket encrypted with SSE-KMS. What should the administrator do to resolve this?

Accepted Answer

Add kms:Decrypt permission to the SageMaker execution role for the KMS key.. The error indicates the execution role lacks kms:Decrypt permission on the KMS key used to encrypt the model artifact. Adding this permission resolves the issue.

Answer

Change the model artifact encryption to SSE-S3.

Answer

Re-upload the model artifact without encryption.

Answer

Attach the AWS managed policy 'AmazonSageMakerFullAccess' to the role.

Question 5

Refer to the exhibit. An IAM policy is attached to a user to allow invoking a SageMaker endpoint. A developer tries to call the endpoint from a laptop with IP 203.0.113.5 and receives an access denied error. What is the most likely reason?

Accepted Answer

The condition restricts the IP address to the 10.0.0.0/8 range.. The policy includes a condition that restricts the source IP to the 10.0.0.0/8 range. The developer's laptop IP (203.0.113.5) is outside this range, causing access denied.

Answer

The resource ARN is incorrect.

Answer

The user does not have permission to assume the SageMaker role.

Answer

The policy does not include access to the API action.

Question 6

A company uses Amazon SageMaker to deploy a real-time inference endpoint. They notice increased latency in predictions during peak hours. Which should they investigate first to address the issue?

Accepted Answer

Review the endpoint auto-scaling policy. Option B is correct because latency typically increases when the endpoint is under-provisioned; auto-scaling policies control scaling behavior. Option A is about training, not inference. Option C is unrelated to inference latency. Option D may affect latency but is not the first thing to investigate.

Answer

Check the data labeling job status

Answer

Modify the training instance type

Answer

Increase the model artifact size

Question 7

A company trains a model daily using Amazon SageMaker and uses the model for real-time inference. They want to detect data drift between the training data and the inference data to decide when to retrain. Which AWS service should they use for this purpose?

Accepted Answer

Amazon SageMaker Model Monitor. Option B is correct because Amazon SageMaker Model Monitor can detect data drift by comparing inference data against a baseline created from training data. Option A is for ETL, not drift detection. Option C is for serverless compute. Option D is for querying data, not monitoring drift.

Answer

Amazon Athena

Answer

AWS Glue

Answer

AWS Lambda

Question 8

A company deploys a machine learning model as a SageMaker real-time endpoint. They need to implement a mechanism to automatically roll back to the previous model version if performance degrades after a deployment. Which approach should they use?

Accepted Answer

Configure the SageMaker endpoint deployment with traffic shifting and set up CloudWatch alarms to trigger automatic rollback. Option B is correct because SageMaker endpoint update with traffic shifting and automatic rollback based on CloudWatch alarms can be configured. Option A requires manual intervention. Option C is complex and not native. Option D is for EC2, not SageMaker.

Answer

Manually update the endpoint to point to the previous model version

Answer

Create multiple endpoints and use Amazon Route 53 weighted routing to shift traffic

Answer

Use AWS CodeDeploy with Amazon EC2 instances behind an Elastic Load Balancer

Question 9

A company wants to ensure that only authorized users and services can invoke a SageMaker real-time endpoint. Which AWS service can be used to manage access control?

Accepted Answer

AWS Identity and Access Management (IAM). Option A is correct because AWS IAM is used to control access to AWS resources, including SageMaker endpoints. Options B, C, and D are for auditing are other purposes.

Answer

Amazon CloudWatch

Answer

AWS CloudTrail

Answer

AWS Config

Question 10

A team deploys a model with SageMaker and notices that the model returns inconsistent results during inference. They suspect a mismatch in feature transformation between the training pipeline and the inference pipeline. Which SageMaker feature can help compare the feature distributions?

Accepted Answer

Amazon SageMaker Model Monitor. Option B is correct because SageMaker Model Monitor can track feature distributions over time and detect drift. Option A is for debugging training jobs. Option C is for explainability and bias detection. Option D is for automated model building.

Answer

Amazon SageMaker Autopilot

Answer

Amazon SageMaker Clarify

Answer

Amazon SageMaker Debugger

Question 11

A company uses Amazon SageMaker Ground Truth to create a labeled dataset. They want to monitor the accuracy of human labelers during the labeling process. Which metric should they track?

Accepted Answer

Accuracy against blinded ground truth. Option C is correct because tracking accuracy against blinded ground truth (known as audit tasks) is the standard way to measure labeler performance. Options A and B are operational metrics not directly measuring accuracy. Option D is not directly accuracy.

Answer

Labeling job cost

Answer

Number of tasks completed

Answer

Task acceptance rate

Question 12

A company stores its model training data in Amazon S3. To meet compliance requirements, all data in transit between the S3 bucket and SageMaker must be encrypted. What should the company enforce?

Accepted Answer

Enforce HTTPS for all S3 access. Option C is correct because enforcing HTTPS for all S3 access ensures that data in transit between the S3 bucket and SageMaker is encrypted using TLS. This meets the compliance requirement for encrypting data in transit, as HTTPS uses TLS to protect data as it travels over the network.

Answer

Enable S3 versioning

Answer

Enable S3 access logging

Answer

Use S3 server-side encryption (SSE-S3)

Question 13

A team uses AWS Auto Scaling for a SageMaker real-time endpoint. They notice that when scaling in, the latest instance is always terminated first, causing disruption to recent requests. How can they configure the scaling policy to terminate the oldest instance first?

Accepted Answer

Configure the termination policy as 'OldestInstance'. Option B is correct because Application Auto Scaling allows setting a termination policy. Option A is for cleanup actions. Option C is for infrastructure as code. Option D is incorrect; the default may not be oldest instance.

Answer

No action needed; this is the default behavior

Answer

Use lifecycle hooks

Answer

Use AWS CloudFormation to manage the endpoint

Question 14

A company deploys a SageMaker model using AWS KMS for encryption at rest. They have a compliance requirement to rotate the KMS key every year without causing downtime for the inference endpoint. Which approach should they take?

Accepted Answer

Enable automatic key rotation on the existing KMS key. Option B is correct because AWS KMS supports automatic key rotation, which rotates the key material yearly without requiring any changes to the endpoint. Options A, C, and D would cause downtime or are unnecessary.

Answer

Use AWS Certificate Manager (ACM) for encryption

Answer

Create a new KMS key and update the endpoint configuration

Answer

Manually rotate the key by recreating the endpoint

Question 15

A company uses Amazon SageMaker Model Monitor to track data quality. The monitoring job triggers an alert indicating that the data distribution has shifted beyond the configured threshold. Which TWO actions should the team take? (Choose TWO.)

Accepted Answer

Update the Model Monitor baseline if the drift is acceptable. Options A and B are correct because the team should retrain the model on the new data distribution if the drift is significant, or update the baseline if the drift is acceptable and represents expected behavior. Options C, D, and E are not appropriate immediate actions.

Answer

Delete the monitoring schedule

Answer

Increase the instance count of the endpoint

Answer

Evaluate the data quality report