MLA-C01 ML Solution Monitoring, Maintenance and Security Practice Test 2 — 15 Questions

Question 1

A financial services company deploys a credit risk model using an Amazon SageMaker endpoint with data capture enabled. The model uses a custom container. The compliance team requires that all inference requests and responses are logged to an S3 bucket with server-side encryption using AWS KMS. The IAM role for the endpoint has the following policy. What must be added to meet the compliance requirement?

Accepted Answer

Add kms:GenerateDataKey and kms:Decrypt permissions to the IAM role.. The correct answer is A because the IAM role for the SageMaker endpoint needs permissions to generate a data key (kms:GenerateDataKey) for encrypting captured data and to decrypt (kms:Decrypt) the KMS key when writing to the S3 bucket. Without these, the endpoint cannot use the customer-managed KMS key for server-side encryption, even if the bucket policy allows it.

Answer

Add s3:PutObjectAcl permission to the IAM role.

Answer

Enable S3 default encryption on the bucket.

Answer

Modify the container to handle encryption internally.

Question 2

Which TWO actions are recommended best practices for securing an Amazon SageMaker notebook instance? (Select TWO.)

Accepted Answer

Use AWS KMS to encrypt the notebook instance's storage volume.. Option C is correct because encrypting the notebook instance's storage volume with AWS KMS ensures data-at-rest protection, which is a fundamental security best practice. SageMaker notebook instances use Amazon EBS volumes for storage, and KMS encryption safeguards sensitive code, datasets, and model artifacts stored on that volume against unauthorized access.

Answer

Use network ACLs to restrict API calls to the SageMaker API.

Answer

Enable Multi-AZ deployment for the notebook instance.

Answer

Associate the notebook instance with a public subnet that has an internet gateway.

Question 3

Which THREE components are required to set up automated model retraining in response to performance degradation using Amazon SageMaker? (Select THREE.)

Accepted Answer

A CloudWatch alarm that triggers when a quality metric falls below a threshold.. Option B is correct because a CloudWatch alarm can monitor a SageMaker Model Monitor quality metric (e.g., accuracy, precision) and trigger an alarm when the metric falls below a defined threshold. This alarm acts as the event source to initiate automated retraining, forming the monitoring and alerting backbone of the retraining pipeline.

Answer

An Amazon SNS topic with a subscription to send a manual approval email.

Answer

A production variant with a canary traffic shift configuration.

Question 4

A company operates an e-commerce platform that uses a machine learning model to recommend products to users. The model is deployed on an Amazon SageMaker endpoint with automatic scaling enabled based on average CPU utilization. The model was trained on historical data and is updated weekly. Recently, the platform experienced a flash sale event that caused a sudden spike in traffic. During the event, the endpoint's latency increased dramatically, and many requests timed out. After the event, the team reviews the CloudWatch metrics and notices that the CPU utilization never exceeded 70%, and the scaling policy was triggered but instances took several minutes to become available. The team wants to prevent similar issues in future flash sales. Which course of action would be MOST effective?

Accepted Answer

Implement scheduled scaling to add capacity ahead of known flash sales.. Option D is correct because scheduled scaling allows you to proactively add capacity ahead of known traffic events like flash sales, eliminating the cold-start delay that occurs when reactive scaling policies (like those based on CPU utilization) must launch new instances. During the flash sale, the scaling policy was triggered but instances took minutes to become available, causing timeouts; scheduled scaling pre-warms the endpoint by adjusting the desired instance count before the traffic spike hits.

Answer

Use predictive scaling based on historical traffic patterns.

Answer

Lower the CPU utilization threshold for the scaling policy to 40%.

Answer

Switch to larger instance types to handle higher CPU loads.

Question 5

A healthcare company deploys a model that predicts patient readmission risk. The model is deployed using a SageMaker real-time endpoint with data capture enabled. The compliance team requires that all inference data be encrypted at rest in S3 using AWS KMS with a customer managed key. The team has configured the endpoint to use an IAM role that includes the necessary KMS permissions. However, after deployment, the captured data is not being written to the S3 bucket. The team checks the CloudWatch logs for the endpoint and finds no errors. The S3 bucket policy is as follows:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::my-bucket/*",
      "Condition": {
        "Bool": {
          "aws:SecureTransport": "false"
        }
      }
    }
  ]
}

The bucket also has a default KMS key. What is the MOST likely reason that the captured data is not being written?

Accepted Answer

The KMS key policy does not grant the SageMaker execution role the kms:GenerateDataKey permission.. The correct answer is C because SageMaker data capture encrypts captured data at rest in S3 using server-side encryption with AWS KMS (SSE-KMS). When a customer managed KMS key is used, the SageMaker execution role must have the kms:GenerateDataKey permission to encrypt the data before writing it to S3. Even if the IAM role has other KMS permissions, without kms:GenerateDataKey, the data capture write operation fails silently, and CloudWatch logs may not show errors because the failure occurs at the KMS encryption step before the S3 PutObject call.

Answer

The bucket policy includes an explicit deny that overrides any allow.

Answer

The bucket policy denies all PutObject requests because aws:SecureTransport is false.

Answer

The S3 bucket does not exist.

Question 6

A data science team deploys a real-time inference endpoint on Amazon SageMaker. They want to monitor for data drift in the input features over time. Which AWS service should they use to capture and analyze the input data distribution?

Accepted Answer

Amazon SageMaker Model Monitor. Amazon SageMaker Model Monitor is designed to detect data drift by capturing and analyzing input data distributions. CloudWatch Logs is for logging, CloudTrail for API auditing, and Athena for ad-hoc querying.

Answer

Amazon Athena

Answer

AWS CloudTrail

Answer

Amazon CloudWatch Logs

Question 7

A company uses SageMaker endpoints with auto-scaling. The endpoint is experiencing high latency during peak hours. The metrics show CPU utilization is low but memory is high. What is the most likely cause?

Accepted Answer

The auto-scaling policy is based on CPU utilization, which does not trigger scaling.. The auto-scaling policy is likely based on CPU utilization, which does not trigger scaling during memory pressure. Memory leaks could be a secondary cause, but the primary issue is the scaling metric.

Answer

The model is not optimized for inference, causing memory leaks.

Answer

The instance type has insufficient network bandwidth.

Answer

The endpoint is deployed in a VPC without a NAT gateway.

Question 8

An ML team trained a model using SageMaker and stored the model artifacts in S3 with server-side encryption using AWS KMS (SSE-KMS). They need to deploy the model to a SageMaker endpoint that uses a different KMS key for inference data encryption. What must they do to ensure the endpoint can decrypt the model artifacts?

Accepted Answer

Grant the SageMaker execution role access to both KMS keys.. The SageMaker execution role must have kms:Decrypt permission on the KMS key used for model artifacts, and also kms:Encrypt and kms:Decrypt on the key for inference data. Providing the same key is not required.

Answer

Provide the same KMS key for both model artifacts and inference data.

Answer

Use a customer-managed key (CMK) with the same key material.

Answer

Configure the endpoint to use SSE-S3 instead of SSE-KMS.

Question 9

A company wants to audit all API calls made to SageMaker endpoints for security compliance. Which AWS service should they enable?

Accepted Answer

AWS CloudTrail. AWS CloudTrail records all API calls for auditing. GuardDuty is for threat detection, Macie for sensitive data discovery, and Config for configuration changes.

Answer

Amazon GuardDuty

Answer

AWS Config

Answer

AWS CloudTrail

Question 10

A machine learning model is deployed on SageMaker and its predictions are used in a production application. The model's accuracy has degraded over time. What is the most likely cause?

Accepted Answer

The model experienced concept drift.. Concept drift occurs when the statistical properties of the target variable change over time, leading to accuracy degradation. Training data shuffling and instance size affect training performance and latency, not accuracy post-deployment.

Answer

The training data was not shuffled properly.

Answer

The model was not compiled for inference.

Answer

The endpoint instance type is too small.

Question 11

A company uses SageMaker training jobs that need to access data in an S3 bucket in a different AWS account. The bucket uses a bucket policy that allows access only from a specific VPC. How should they configure the training job?

Accepted Answer

Use cross-account access with an IAM role and add a bucket policy allowing the training job's VPC.. The training job should be launched in a VPC with an S3 VPC endpoint, and the bucket policy must allow the VPC. Additionally, an IAM role in the source account with cross-account trust is needed. Option C combines both requirements.

Answer

Use AWS DataSync to copy data to the training account's S3.

Answer

Create an IAM role in the source account and assume it from the training account.

Answer

Use an S3 VPC endpoint in the training job's VPC and attach a bucket policy that allows the VPC.

Question 12

A team wants to automatically retrain a model when new labeled data arrives. Which SageMaker feature can orchestrate this workflow?

Accepted Answer

SageMaker Pipelines. SageMaker Pipelines is a workflow orchestration service that can automate retraining pipelines. Model Monitor detects drift, Debugger debugs training, and Autopilot automates model building.

Answer

SageMaker Model Monitor

Answer

SageMaker Debugger

Answer

SageMaker Autopilot

Question 13

A model deployed on a SageMaker endpoint is returning predictions. The team wants to log all predictions to an S3 bucket for auditing. What is the most efficient way to achieve this?

Accepted Answer

Enable SageMaker endpoint data capture to the S3 bucket.. SageMaker data capture is designed for this purpose and can be enabled on the endpoint configuration to automatically capture input and output data to S3. Modifying inference code is custom and less efficient, Firehose adds complexity, and CloudWatch Logs export is for logs.

Answer

Configure CloudWatch Logs to export to S3.

Answer

Modify the inference code to write logs to S3.

Answer

Use Amazon Kinesis Data Firehose to stream predictions to S3.

Question 14

A company wants to restrict access to a SageMaker notebook instance so that only a specific IAM role can open the notebook via JupyterLab. The notebook instance is associated with a lifecycle configuration that installs custom packages. What is the correct way to enforce access control?

Accepted Answer

Grant the specific IAM role permission to call sagemaker:CreatePresignedNotebookInstanceUrl on that notebook instance.. The specific IAM role should be granted permission to call the `sagemaker:CreatePresignedNotebookInstanceUrl` action for that notebook instance ARN. Other options like VPC or Systems Manager do not control who can open the notebook.

Answer

Set the notebook instance's Direct Internet Access to disabled and use IAM authentication.

Answer

Use AWS Systems Manager to proxy SSH access, then use IAM permission.

Answer

Configure the notebook instance to use a VPC and restrict access via security groups.

Question 15

A team uses SageMaker Ground Truth to create labeled datasets. They need to ensure labeling jobs are cost-effective. Which TWO measures should they take? (Select TWO.)

Accepted Answer

Set up a labeling workflow with 'Incremental training'.. Automated data labeling reduces manual labeling cost by using model predictions to label data, and incremental training reduces the number of items that need manual labeling by starting from an existing model.

Answer

Use a smaller instance type for the labeling job.

Answer

Use a smaller workforce type.

Answer

Enable the 'Consolidated billing' for labeling costs.