MLA-C01 Practice Test 28 — 15 Questions

Question 1

A machine learning engineer at a retail company is monitoring a production model that predicts inventory demand. The model's prediction accuracy has dropped significantly over the past week. The engineer checks the model's input data and notices a new product category was introduced with a different distribution. Which concept is most likely causing the performance degradation?

Accepted Answer

Covariate shift. B is correct because covariate shift occurs when the distribution of the input features changes while the relationship between features and the target remains the same. In this scenario, the introduction of a new product category with a different distribution alters the input data distribution, causing the model to encounter unseen patterns and degrade in prediction accuracy.

Answer

Concept drift

Answer

Data leakage

Answer

Model decay

Question 2

A data science team has trained a PyTorch model using Amazon SageMaker and wants to deploy it with a custom inference container that includes a pre-processing step. The team needs to minimize latency and ensure the pre-processing runs only once per request. Which SageMaker real-time inference option should they use?

Accepted Answer

Create a SageMaker inference pipeline with two containers: one for pre-processing and one for inference.. Option D is correct because a SageMaker inference pipeline allows you to chain two containers in a single endpoint, where the first container handles pre-processing and the second runs inference. This ensures that pre-processing runs exactly once per request, minimizing latency by avoiding redundant processing and keeping the request within the same HTTP connection.

Answer

Deploy the model on a multi-model endpoint and include pre-processing in the model code.

Answer

Use a batch transform job with a pre-processing script.

Answer

Package pre-processing and inference in a single container with a custom entry point.

Question 3

A data scientist is trying to create a SageMaker endpoint configuration with 6 instances of ml.c5.large for a production variant. The creation fails with the error shown in the exhibit. Which action should the data scientist take to resolve this issue?

Accepted Answer

Request a service quota increase for ml.c5.large for real-time endpoints from the AWS Service Quotas console.. The error indicates that the requested number of instances exceeds the service quota for ml.c5.large for real-time endpoints. AWS enforces default limits on instance counts per instance type per region. Requesting a quota increase via the Service Quotas console is the correct action to raise the limit and allow the deployment of 6 instances.

Answer

Create two separate endpoint configurations, each with 3 instances, and distribute traffic between them.

Answer

Use a different instance type, such as ml.m5.large, which has a higher limit.

Answer

Delete unused endpoints to free up resources.

Question 4

A company is using Amazon SageMaker to host a real-time inference endpoint. They want to restrict access to the endpoint to only a specific VPC and require authentication using AWS IAM. Which TWO configuration steps should they take to achieve this? (Choose TWO.)

Accepted Answer

Configure the endpoint to be deployed in a private subnet within the VPC. Option A is correct because deploying the SageMaker endpoint in a private subnet within the VPC ensures that the endpoint is not publicly accessible and can only be reached from within that VPC. This is achieved by using a VPC interface endpoint (AWS PrivateLink) or by placing the endpoint directly in the VPC, which restricts network traffic to the VPC boundary.

Answer

Attach a resource-based policy to the endpoint that denies all traffic except from the VPC

Answer

Place the endpoint behind Amazon CloudFront to act as a proxy

Answer

Use a public subnet and configure a security group to allow only the company's IP range

Question 5

A team is deploying a machine learning model using Amazon SageMaker. They need to serve predictions with sub-100ms latency for a real-time application. The model is a large ensemble that requires 4 GB of memory. The team expects traffic of 100 requests per second initially, but it may double during peak hours. Which instance type and deployment configuration should the team choose to minimize cost while meeting the latency requirement?

Accepted Answer

Deploy on one ml.c5.large instance with an Application Auto Scaling target tracking policy based on memory utilization. Option A is correct because the ml.c5.large instance provides 4 GB of memory, which meets the model's requirement, and its compute-optimized nature ensures low-latency inference. Using Application Auto Scaling with a target tracking policy based on memory utilization allows the instance to scale out during traffic spikes (up to 200 requests per second) while minimizing cost by running a single instance during normal load.

Answer

Deploy on one ml.t2.medium instance with an Application Auto Scaling target tracking policy based on CPU utilization

Answer

Deploy on one ml.p3.2xlarge instance with provisioned concurrency

Answer

Deploy on two ml.m5.large instances behind a load balancer with manual scaling

Question 6

A machine learning engineer is deploying a PyTorch model for real-time inference on SageMaker. The model requires GPU for low-latency predictions. The deployment fails with the error: 'The primary container does not support the requested instance type.' The instance type is ml.p3.2xlarge. Which action should the engineer take to resolve the issue?

Accepted Answer

Verify that the PyTorch framework version specified in the SageMaker estimator matches a version that supports GPU instances. Option C is correct because the error 'The primary container does not support the requested instance type' typically occurs when the specified PyTorch framework version in the SageMaker estimator does not include GPU support for the chosen instance type (ml.p3.2xlarge). SageMaker's prebuilt PyTorch containers are version-specific and only certain versions are compiled with CUDA and GPU libraries; using a version that lacks GPU support causes the container to reject GPU instance types. Verifying and selecting a PyTorch version that explicitly supports GPU instances resolves the mismatch.

Answer

Use SageMaker Neo to compile the model for the target instance type

Answer

Request a service quota increase for the ml.p3.2xlarge instance type

Answer

Create a custom inference container and use it with the SageMaker model

Question 7

A company is using Amazon SageMaker to deploy a model for real-time inference. The model requires access to a private S3 bucket that contains reference data. The company wants to ensure that the endpoint can access the S3 bucket without using a public internet connection. Which TWO actions should they take? (Select TWO.)

Accepted Answer

Attach the endpoint to a VPC that has a VPC endpoint for S3.. Option B is correct because attaching the SageMaker endpoint to a VPC with a VPC endpoint for S3 (Gateway type) allows the endpoint to access the S3 bucket using AWS's private network, bypassing the public internet. This ensures traffic stays within the AWS backbone, meeting the requirement for no public internet connection. Option C is also correct because the SageMaker execution role must have an IAM policy with s3:GetObject permissions to authorize the read access to the private S3 bucket, which is a prerequisite for any S3 operation.

Answer

Configure the endpoint's security group to allow outbound traffic to the S3 bucket's IP range.

Answer

Attach the endpoint to a VPC with an internet gateway and route the S3 traffic through the internet gateway.

Answer

Attach the endpoint to a VPC with a NAT gateway to route traffic to S3.

Question 8

A machine learning team is building a CI/CD pipeline for model deployment using Amazon SageMaker. They need to ensure that all model artifacts are encrypted at rest and in transit, and that access to the models is controlled via IAM. Which TWO actions should the team take to meet these requirements? (Choose TWO.)

Accepted Answer

Configure the SageMaker notebook instance to use a KMS key for encryption. Option D is correct because configuring a SageMaker notebook instance to use a KMS key ensures that data at rest on the notebook's storage volume (e.g., EBS) is encrypted. This directly addresses the requirement for encryption at rest for model artifacts during development. Option E is correct because using HTTPS endpoints for invoking the SageMaker model ensures encryption in transit via TLS, protecting data as it moves between clients and the model endpoint.

Answer

Set the SageMaker model's 'EnableNetworkIsolation' parameter to true

Answer

Enable default encryption on the S3 bucket that stores model artifacts

Answer

Enable AWS CloudTrail to log all API calls to SageMaker

Question 9

A machine learning engineer has configured a SageMaker Model Monitor schedule for data quality monitoring as shown in the exhibit. The schedule is set to run hourly. However, the engineer notices that the monitoring jobs are not producing output in the specified S3 bucket. What is the most likely cause?

Accepted Answer

The DataAnalysisStartTime and DataAnalysisEndTime are set to a past date, so no data is analyzed.. Option B is correct because the DataAnalysisStartTime and DataAnalysisEndTime parameters define the time window for which SageMaker Model Monitor analyzes data. When both are set to a past date that has already passed, the monitoring job finds no new data to analyze within that window, resulting in no output being written to the S3 bucket. The schedule runs hourly, but the analysis window is fixed to a historical period, so each execution produces no results.

Answer

The output_path is incorrectly placed; it should be under the MonitoringOutputConfig.

Answer

The MonitoringType should be 'ModelQuality' to enable data quality monitoring.

Answer

The cron expression is incorrectly formatted for an hourly schedule.

Question 10

A data science team has trained a model using SageMaker and wants to deploy it for real-time inference with automatic scaling based on request latency. The deployment must handle unpredictable traffic spikes without manual intervention. Which combination of SageMaker features should the team use?

Accepted Answer

Create a SageMaker endpoint with an Application Auto Scaling target tracking policy based on the SageMakerVariantInvocationsPerInstance metric. Option A is correct because it uses a SageMaker endpoint with an Application Auto Scaling target tracking policy based on the SageMakerVariantInvocationsPerInstance metric. This allows the endpoint to automatically scale the number of instances in response to changes in request latency, as the metric directly reflects the load per instance. The target tracking policy adjusts capacity to maintain a target value for the metric, handling unpredictable traffic spikes without manual intervention.

Answer

Deploy the model on a multi-model endpoint and manually adjust the number of instances via the AWS Management Console

Answer

Deploy the model on an Elastic Inference accelerator and use AWS Auto Scaling with a scheduled policy

Answer

Create a batch transform job with a scheduled Lambda function to trigger scaling

Question 11

A company is using SageMaker Pipelines to automate a multi-step ML workflow. The pipeline includes data preprocessing, training, and model evaluation. The team wants to ensure that if the evaluation step fails, the pipeline stops and sends an alert to the operations team. Which SageMaker Pipelines feature should they use?

Accepted Answer

Use a Condition step to check the evaluation result and route to a Fail step if the result indicates failure. Option D is correct because SageMaker Pipelines provides a built-in Condition step that evaluates a boolean expression (e.g., checking if evaluation metrics meet a threshold) and then routes execution to different steps. If the condition fails, you can direct the pipeline to a Fail step, which immediately stops the pipeline and marks it as failed. This is the native, event-driven way to halt a pipeline based on step output without relying on external services.

Answer

Configure an Amazon CloudWatch Events rule to monitor the pipeline execution status and stop it if the evaluation step fails

Answer

Register the model in the Model Registry only if evaluation passes, and configure the pipeline to stop if registration fails

Answer

Add a Lambda step after the evaluation step that checks the evaluation metrics and sends an SNS notification if the metrics are below a threshold

Question 12

A data science team is using Amazon SageMaker to train and deploy a binary classification model. They want to continuously monitor the model for data drift in production. Which combination of AWS services and SageMaker features should they use to implement automated drift detection with minimal operational overhead?

Accepted Answer

SageMaker Model Monitor and Amazon CloudWatch. SageMaker Model Monitor is the native SageMaker feature designed specifically for continuously monitoring deployed models for data drift, bias drift, and feature attribution drift. It automatically captures inference requests and responses, computes statistics, and publishes metrics to Amazon CloudWatch, which can trigger alarms for drift detection. This combination provides automated drift detection with minimal operational overhead because it requires no custom infrastructure or manual scheduling.

Answer

SageMaker Debugger and Amazon SNS

Answer

SageMaker Pipelines and AWS Lambda

Answer

SageMaker Clarify and AWS Config

Question 13

A machine learning engineer deploys a model to an Amazon SageMaker endpoint with data capture enabled. The endpoint uses a production variant with initial instance count of 2. After a week, they notice that the captured data is not being sent to the specified Amazon S3 bucket. The IAM role used by the endpoint has the following policy attached. What is the MOST likely reason for the failure?

Accepted Answer

The IAM role does not have permission to write to the correct S3 prefix.. Option C is correct because the IAM role attached to the SageMaker endpoint must have write permissions to the exact S3 prefix where data capture is configured. The policy shown likely grants access to a broader bucket or a different prefix, but not the specific path (e.g., s3://bucket-name/prefix/) that the endpoint's DataCaptureConfig specifies. Without s3:PutObject on that exact prefix, the captured data fails to upload silently.

Answer

The S3 bucket does not exist.

Answer

The S3 bucket uses AWS KMS encryption and the role lacks kms:Decrypt permission.

Answer

The IAM role does not have s3:ListBucket permission.

Question 14

A data science team deploys a regression model using Amazon SageMaker. After one week, the model's prediction accuracy drops significantly. The team needs to detect this degradation automatically and trigger retraining. Which AWS service should they use to monitor the model's performance over time and set up alerts?

Accepted Answer

Amazon SageMaker Model Monitor. Amazon SageMaker Model Monitor is the correct choice because it is purpose-built to continuously monitor machine learning models deployed on SageMaker endpoints for data drift, feature attribution drift, and prediction quality degradation. It automatically compares live inference data against a baseline, triggers alerts when performance drops, and can be configured to initiate retraining pipelines via AWS Lambda or Step Functions, directly addressing the need to detect accuracy degradation and trigger retraining.

Answer

AWS CloudWatch

Answer

Amazon Inspector

Answer

AWS Config

Question 15

A company is using Amazon SageMaker to train a model on sensitive customer data. The security team requires that all data be encrypted in transit and at rest, and that the training job does not have internet access. Which configuration should the team use to meet these requirements?

Accepted Answer

Configure the training job to run in a private subnet with no internet access, and use a KMS key for encryption. Option C is correct because running the SageMaker training job in a private subnet with no internet access ensures the job cannot reach the public internet, satisfying the no-internet-access requirement. Using an AWS KMS key for encryption at rest (for the S3 bucket and EBS volumes) and enforcing encryption in transit (via HTTPS/TLS for SageMaker and S3 endpoints) meets the encryption requirements. SageMaker training jobs in a private subnet use VPC endpoints (e.g., S3 and SageMaker API endpoints) to communicate securely without internet access.

Answer

Configure the training job to run in a public subnet with a security group that blocks outbound traffic

Answer

Configure the training job to run in a private subnet, but disable encryption to reduce latency

Answer

Configure the training job to run in a VPC with a NAT gateway, and use default SageMaker encryption