MLA-C01 ML Solution Monitoring, Maintenance and Security Practice Test 6 — 15 Questions

Question 1

A machine learning team is setting up Model Monitor for a deployed model. Which THREE factors should they consider when configuring the monitoring schedule? (Select three.)

Accepted Answer

The monitoring job can be configured to send notifications via Amazon SNS.. Options B, D, and E are correct. B: Sufficient sample size ensures statistical significance. D: Schedule should align with data arrival patterns to detect drift promptly. E: SNS notifications can be set up for alerts. A is not necessarily correct; frequency depends on data volume. C is incorrect because constraints are updated manually or via baseline jobs, not automatically after each run.

Answer

The frequency of monitoring should be at least daily.

Answer

The constraints file must be updated after each monitoring run.

Question 2

A data scientist wants to monitor a deployed model for performance degradation. Which TWO metrics from Amazon CloudWatch should they use to detect issues? (Select two.)

Accepted Answer

ModelLatency. Options A and D are correct. A: ModelLatency can indicate if the model is slowing down due to code changes or resource constraints. D: Invocation5XXErrors indicate server-side failures that may signal degradation. B is about volume, not quality. C is not a standard CloudWatch metric. E is about instance health, not model performance.

Answer

ModelQuality

Answer

CpuUtilization

Answer

InvocationCount

Question 3

Refer to the exhibit. A SageMaker endpoint is failing health checks. What is the most likely cause?

Accepted Answer

The inference code has a file descriptor leak.. Option C is correct because the error 'Too many open files' indicates a file descriptor leak in the inference code. Option A would show memory errors. Option B would show network errors. Option D would show incorrect port errors.

Answer

The endpoint is not correctly configured with VPC settings.

Answer

The model is too large for the instance memory.

Answer

The model server is using an incorrect port.

Question 4

Refer to the exhibit. A SageMaker training job using this IAM role fails with an access denied error when trying to read a file from s3://my-bucket/training-data/model_input.csv. However, a different file at s3://my-bucket/training-data/input/data.csv can be read successfully. What is the most likely reason?

Accepted Answer

The IAM policy restricts access to objects only under the 'training-data/' prefix using an incorrect condition key.. Option D is correct because the condition 's3:prefix' evaluates to the object’s key prefix, and 'training-data/model_input.csv' does not start with 'training-data/'? Actually it does, but the condition 'StringEquals' requires an exact match? The s3:prefix condition key is used with 'StringLike' or 'StringEquals' to match the prefix. With 'StringEquals', it must exactly equal 'training-data/', but the object key is 'training-data/model_input.csv' which starts with that prefix. However, the condition is evaluated against the request's s3:prefix value, not the object key. There is an AWS nuance: s3:prefix condition key checks the prefix used in the request (e.g., in ListObjects), not the object key itself. For GetObject, the condition does not apply because GetObject uses s3:object key. So the policy is misconfigured. The correct condition for GetObject should be on s3:object key using a StringLike. Therefore, the most likely reason is that the policy condition is incorrectly applied. Option A is plausible but less likely since the other file works. Option B is about KMS, which would affect both files. Option C is about bucket policy overriding. Option D about file name containing special characters is unrelated.

Answer

The file model_input.csv is encrypted with a KMS key that the role does not have access to.

Answer

The file name contains special characters that are not encoded correctly.

Answer

The S3 bucket has a bucket policy that denies access to the specific file.

Question 5

Refer to the exhibit. A team configured a SageMaker Model Monitor schedule for data quality. The baseline was created from a training dataset. After running for a day, the monitoring results show frequent violations. What is the most likely cause?

Accepted Answer

The baseline was created from a dataset that does not represent production data.. Option A is correct because the baseline from training data may not represent the production data distribution, causing frequent violations. Option B is not likely because hourly monitoring is typical. Option C would cause job failures, not violations. Option D would cause timeout, not violations.

Answer

The environment variable max_runtime_in_seconds is too low.

Answer

The schedule runs too often (every hour), causing overload.

Answer

The monitoring output destination is incorrect.

Question 6

A data science team deploys a regression model to Amazon SageMaker for real-time inference. After one month, the model's prediction errors increase significantly, but data distributions remain unchanged. Which monitoring approach is MOST suitable for detecting this issue?

Accepted Answer

Set up Amazon SageMaker Model Monitor to track model performance metrics against ground truth labels as they arrive.. Option A is correct because the model's prediction errors increased despite no data drift, indicating concept drift. Model performance monitoring compares predictions against ground truth. Option B is wrong because data drift monitors input distribution changes, not concept drift. Option C is wrong because feature attribution drift is a type of data drift. Option D is wrong because model latency is a performance metric, not accuracy.

Answer

Use Amazon SageMaker Clarify to monitor feature attribution drift.

Answer

Enable Amazon CloudWatch to monitor model endpoint latency.

Answer

Configure Amazon SageMaker Model Monitor to track data drift on the input features.

Question 7

A company uses an Amazon SageMaker endpoint for real-time inference. The security team requires that all traffic between the endpoint and the client application be encrypted in transit. Which configuration ensures this?

Accepted Answer

The endpoint is automatically served over HTTPS; no additional configuration is needed.. Option B is correct because SageMaker endpoints are already HTTPS-enabled by default, providing encryption in transit via TLS. Option A is wrong because a VPC does not provide encryption in transit by default; it provides network isolation. Option C is wrong because SageMaker does not directly use AWS Certificate Manager for endpoint encryption; it uses AWS-managed certificates. Option D is wrong because AWS KMS is for encryption at rest, not in transit.

Answer

Deploy the endpoint in a VPC and use VPC Endpoints.

Answer

Use AWS Key Management Service (KMS) to encrypt the data in transit.

Answer

Attach an AWS Certificate Manager (ACM) certificate to the endpoint.

Question 8

A machine learning team deploys a custom container image for an Amazon SageMaker training job. The container needs to access an S3 bucket that contains sensitive data. The team wants to follow the principle of least privilege. How should the team grant access?

Accepted Answer

Create an IAM role with S3 access and assign it as the SageMaker execution role for the training job.. Option C is correct because SageMaker execution role assigned to the training job is the best practice. Option A is wrong because hardcoding keys is insecure. Option B is wrong because instance profile is for EC2, not SageMaker training jobs directly; SageMaker uses execution roles. Option D is wrong because SageMaker does not support S3 bucket policies with principal as the training job ARN directly; the execution role is used.

Answer

Attach an IAM instance profile to the training instance with permissions to the bucket.

Answer

Configure an S3 bucket policy that grants access to the training job's ARN.

Answer

Store AWS access keys in the container image and use them to access the bucket.

Question 9

A company has a batch transform job in Amazon SageMaker that processes large datasets every night. Recently, the job has been failing sporadically with an out-of-memory error. The data size has not increased. What is the MOST likely cause?

Accepted Answer

The custom inference code has a memory leak that gradually consumes available memory.. Option D is correct because a memory leak in custom code would cause increasing memory usage over time within a single job, leading to OOM. Option A is wrong because instance type is fixed; if it worked before, instance type is not the issue. Option B is wrong because if the data size hasn't increased, total data is not the cause. Option C is wrong because data distribution change doesn't directly cause OOM; it might cause different processing but not necessarily memory exhaustion.

Answer

The data distribution has shifted, causing different memory usage patterns.

Answer

The instance type is not large enough to handle the dataset.

Answer

The batch transform input data has increased in size.

Question 10

An e-commerce company uses a multi-model endpoint on Amazon SageMaker to serve several deep learning models. After a new model version is deployed, the endpoint starts returning 503 errors for some models. Monitoring shows that the endpoint's memory utilization is near 100%. What should the team do to resolve this issue while minimizing operational overhead?

Accepted Answer

Increase the number of instances for the endpoint and configure an auto-scaling policy based on memory utilization.. Option C is correct because increasing the endpoint's instance count spreads the memory load, and automating instance scaling with a target tracking policy adjusts based on memory. Option A is wrong because SageMaker does not support per-model scaling; scaling is per endpoint. Option B is wrong because moving to single-model endpoints would increase operational overhead and cost. Option D is wrong because Model Monitor doesn't help with scaling.

Answer

Deploy each model on its own separate endpoint to isolate memory usage.

Answer

Use Amazon SageMaker Model Monitor to detect memory leaks and send alerts.

Answer

Use SageMaker's built-in model scaling feature to allocate more memory to the affected model.

Question 11

A machine learning engineer is setting up an Amazon SageMaker notebook instance. The instance needs to access a private S3 bucket that contains training data. The notebook instance is in a VPC. Which combination of steps will grant access to the S3 bucket? (Choose TWO.)

Accepted Answer

Create a VPC endpoint for S3 in the same VPC and subnet.. Options B and D are correct. The notebook needs an IAM role with S3 permissions (B) and a VPC endpoint for S3 (D) to access the bucket privately. Option A is wrong because internet gateway is not needed if using VPC endpoint; using NAT would be more complex. Option C is wrong because assigning public IP is not necessary for private access. Option E is wrong because NAT gateway is not required if using VPC endpoint.

Answer

Assign a public IP address to the notebook instance.

Answer

Set up a NAT gateway in the public subnet.

Answer

Attach an internet gateway to the VPC.

Question 12

A company is using an Amazon SageMaker pipeline for automated retraining. The pipeline fails intermittently due to transient errors in the training job. Which steps should the team take to ensure the pipeline completes successfully? (Choose THREE.)

Accepted Answer

Enable managed spot training for cost savings and use checkpointing to resume from interruptions.. Options A, C, and E are correct. A: Add retry policies for the training step. C: Use spot instances with managed spot training to handle interruptions. E: Implement exponential backoff in custom code for API calls. Option B is wrong because increasing instance count does not solve transient errors; it adds cost. Option D is wrong because SageMaker does not support automatic checkpointing across retries; you need to implement custom checkpointing.

Answer

Use a larger instance type for the training job to reduce the chance of failure.

Answer

Implement automatic model checkpointing by setting the CheckpointConfig in the pipeline step.

Question 13

A financial services company must ensure that all data used by Amazon SageMaker training jobs is encrypted at rest. The company wants to use a customer-managed key (CMK) for the encryption. Which steps are necessary to achieve this? (Choose TWO.)

Accepted Answer

Create a CMK in AWS KMS and add the SageMaker service principal to the key policy to allow it to use the key.. Options A and C are correct. A: Grant SageMaker permissions to use the CMK. C: Specify the CMK in the KmsKeyId parameter of the training job. Option B is wrong because adding to S3 encryption is for S3, not SageMaker. Option D is wrong because enabling S3 default encryption does not cover SageMaker's internal storage. Option E is wrong because SageMaker encrypts at rest by default with AWS-managed keys, but for CMK you specify it.

Answer

Enable SageMaker's default encryption for the training job by setting the EnableDefaultEncryption flag.

Answer

Enable S3 default encryption using the CMK on all buckets containing training data.

Answer

Use CloudWatch Logs encryption to protect the training logs.

Question 14

A team deploys a machine learning model using an Amazon SageMaker endpoint. They need to monitor for data drift and model quality issues. Which AWS services or features should they use? (Choose THREE.)

Accepted Answer

Amazon SageMaker Clarify. Options A, C, and E are correct. A: SageMaker Model Monitor can monitor data drift and model quality. C: SageMaker Clarify can monitor bias and feature attribution drift. E: Amazon CloudWatch can collect custom metrics and set alarms, used with Model Monitor. Option B is wrong because SageMaker Ground Truth is for labeling, not monitoring. Option D is wrong because AWS Glue is for ETL, not monitoring deployed models.

Answer

AWS Glue DataBrew

Answer

Amazon SageMaker Ground Truth

Question 15

Refer to the exhibit. A SageMaker execution role has the IAM policy shown. The team attempts to run a training job that writes results to 's3://my-bucket/training/output/model.tar.gz'. What will happen?

Accepted Answer

The training job will succeed and write the model artifact.. Option C is correct. The Deny statement blocks PutObject on the specific object 'sensitive-data.csv', but the write to 'model.tar.gz' is allowed by the second statement. There is no explicit deny on 'model.tar.gz'. Option A is wrong because the Deny is specific. Option B is wrong because there is no conflict; Deny only applies to that one object. Option D is wrong because the Deny does not affect this write.

Answer

The training job will fail because the Deny statement blocks all PutObject actions.

Answer

The training job will fail because the Deny statement overrides the Allow.

Answer

The training job will succeed, but the output file will be encrypted with a different key.