MLA-C01 Deployment and Orchestration of ML Workflows Practice Test 5 — 15 Questions

Question 1

A company uses Amazon SageMaker to deploy a model for real-time inference. They want to perform A/B testing between two model versions. Which TWO actions should the company take to set up A/B testing? (Choose TWO.)

Accepted Answer

Create an endpoint configuration with multiple production variants, each with a different model.. Option A is correct because in SageMaker, A/B testing between two model versions is achieved by creating an endpoint configuration with multiple production variants, each pointing to a different model. This allows the endpoint to host both models simultaneously and route traffic between them based on assigned weights.

Answer

Use Amazon CloudWatch Evidently to split traffic between models.

Answer

Enable auto scaling for each production variant individually.

Answer

Set the second production variant's weight to 0 and update later to 100.

Question 2

A machine learning team is building a CI/CD pipeline to train and deploy models using Amazon SageMaker. They want to ensure that the deployment step only proceeds if the model evaluation metrics exceed a certain threshold. Which THREE components should the team include in the pipeline? (Choose THREE.)

Accepted Answer

The SageMaker Model Registry to approve and store the model after evaluation.. Option C is correct because the SageMaker Model Registry is the central component for approving and storing model versions after evaluation. It enables governance by allowing you to set approval statuses (e.g., Approved, Rejected) and track model lineage, ensuring only validated models proceed to deployment.

Answer

An AWS Lambda function for manual approval.

Answer

An AWS CodeBuild project to compile the model artifacts.

Question 3

A data science team is deploying a model on Amazon SageMaker and wants to protect the endpoint from unauthorized access. Which TWO methods can the team use to secure the endpoint? (Choose TWO.)

Accepted Answer

Configure the endpoint to be deployed within a VPC and control traffic using security groups and network ACLs.. Option A is correct because deploying a SageMaker endpoint within a VPC allows you to control inbound and outbound traffic using security groups and network ACLs, effectively restricting network-level access to the endpoint. This is a fundamental network security measure that prevents unauthorized network traffic from reaching the endpoint.

Answer

Use a resource-based IAM policy on the endpoint to restrict invocation.

Answer

Place an Amazon API Gateway in front of the endpoint with AWS WAF.

Answer

Attach a security group directly to the SageMaker endpoint.

Question 4

A data science team needs to deploy a frequently updated PyTorch model for real-time inference. The model is retrained weekly and versioned using SageMaker Model Registry. Which deployment strategy minimizes downtime and allows easy rollback?

Accepted Answer

Configure SageMaker endpoints with multiple production variants and use canary deployment to shift traffic gradually.. Option C is correct because SageMaker endpoints with multiple production variants enable canary deployment, which shifts traffic gradually from the old model to the new one. This minimizes downtime by keeping both variants active during the transition and allows easy rollback by simply redirecting all traffic back to the previous variant if issues arise.

Answer

Deploy the model on an EC2 instance behind an Application Load Balancer and manually update the instance with the new model version.

Answer

Deploy the model using AWS Lambda with a container image and trigger via API Gateway.

Answer

Use SageMaker hosting with a single production variant and update the endpoint with a new model configuration each week.

Question 5

An ML team is deploying a model using SageMaker. The model requires GPU inference and must be available in multiple AWS regions for low latency. The team has created a multi-model endpoint with GPU instances. After deployment, they notice high latency spikes when a new model is loaded. What is the most likely cause?

Accepted Answer

The team is using a multi-model endpoint, which loads models on demand; loading a model into GPU memory causes latency spikes.. A multi-model endpoint (MME) loads models on demand from Amazon S3 into the instance's memory. When a new model is requested and not already cached, SageMaker must download the model artifacts and load them into GPU memory, which is a time-consuming operation that causes a latency spike for the first inference request. This cold-start behavior is inherent to MMEs and explains the observed spikes.

Answer

The endpoint is configured with a single production variant, causing all traffic to overload one instance.

Answer

The endpoint is using the wrong instance type that lacks sufficient GPU memory.

Answer

The model is too large for the specified container memory, causing swap to disk.

Question 6

A company is deploying a ML model for real-time fraud detection using SageMaker. The model must process requests within 50 ms and scale to handle up to 10,000 requests per second during peak hours. The data includes PII, so all traffic must stay within a VPC. The team has configured the SageMaker endpoint with a VPC and an internet gateway for model downloads. During a load test, the endpoint fails to achieve the required throughput. Which change would most likely resolve the issue?

Accepted Answer

Use VPC endpoints (interface endpoint for SageMaker and gateway endpoint for S3) to keep traffic within AWS backbone.. The correct answer is B because the endpoint is currently using an internet gateway for model downloads, which forces traffic out to the public internet and back, adding latency and risking throughput failures. By using VPC interface endpoints for SageMaker and gateway endpoints for S3, all traffic stays within the AWS backbone network, reducing network overhead and meeting the 50 ms latency requirement. This also keeps PII traffic within the VPC, satisfying security constraints.

Answer

Remove the VPC configuration and use public endpoints to reduce network overhead.

Answer

Add a NAT gateway to allow the SageMaker endpoint to access the internet efficiently.

Answer

Increase the instance count and use a larger instance type to handle the throughput.

Question 7

A team wants to automate the retraining and deployment of an ML model whenever new labeled data arrives in S3. The workflow includes data preprocessing, training, evaluation, and conditional deployment. Which AWS service is best suited for orchestrating this end-to-end pipeline?

Accepted Answer

Amazon SageMaker Pipelines triggered by S3 events via EventBridge.. Amazon SageMaker Pipelines is purpose-built for ML workflows, offering native integration with SageMaker for training, evaluation, and conditional deployment steps. Triggered by S3 events via Amazon EventBridge, it automates the end-to-end pipeline from data preprocessing to conditional model deployment without requiring custom orchestration code.

Answer

AWS Step Functions with Lambda functions for each step.

Answer

AWS Glue workflows with triggers based on S3 events.

Answer

AWS CodePipeline with source from S3 and build from CodeBuild.

Question 8

An MLOps engineer is setting up a SageMaker endpoint for a model that performs inference on large images. The model is containerized and expects input in a specific format. The team wants to preprocess the images (resize and normalize) before passing them to the model. What is the most efficient way to implement this?

Accepted Answer

Configure SageMaker to use a preprocessing container as the first step of an inference pipeline, followed by the model container.. Option A is correct because SageMaker Inference Pipelines allow you to chain multiple containers in a serial fashion, where the output of one container becomes the input of the next. By placing a preprocessing container as the first step, you can resize and normalize large images before passing them to the model container, which keeps the model container focused on inference and avoids unnecessary data transfer or custom code. This is the most efficient and natively supported approach within SageMaker for multi-step inference workflows.

Answer

Use Amazon API Gateway to perform request transformation before forwarding to the endpoint.

Answer

Package the preprocessing logic into the same Docker container as the model.

Answer

Use a Lambda function as a proxy to preprocess requests before calling the SageMaker endpoint.

Question 9

A company deploys a model using SageMaker real-time endpoint with auto scaling. They observe that during a traffic spike, the endpoint quickly scales up to 10 instances, but after the spike, it takes a long time to scale down, leading to high costs. The scaling policy is based on a simple average CPU utilization threshold. Which adjustment would optimize the scaling down behavior?

Accepted Answer

Decrease the scale-in cooldown period to allow the endpoint to scale down faster when utilization drops.. The correct answer is B because decreasing the scale-in cooldown period allows the endpoint to respond more quickly to sustained drops in CPU utilization. By default, SageMaker auto scaling uses cooldown periods to prevent rapid fluctuations; a long scale-in cooldown delays the termination of instances after utilization falls, keeping costs high. Reducing this cooldown lets the endpoint scale down faster when the spike subsides, directly addressing the problem.

Answer

Increase the scale-in cooldown period to prevent premature scale-down.

Answer

Use a step scaling policy with a larger step adjustment for scale-in.

Answer

Change the scaling policy to use memory utilization instead of CPU.

Question 10

A team wants to apply a custom container for inference on SageMaker. The container needs to implement a web server that responds to API requests. Which protocol and port must the container listen on to be compatible with SageMaker hosting?

Accepted Answer

The container must listen on port 8080 and use HTTP protocol.. SageMaker requires custom inference containers to listen on port 8080 and communicate over HTTP (not HTTPS). The SageMaker hosting service uses a proxy that terminates HTTPS and forwards plain HTTP requests to the container on port 8080. This ensures compatibility with the built-in model serving infrastructure.

Answer

The container must listen on port 8080 and use HTTPS protocol.

Answer

The container can listen on any port as long as the port is specified in the endpoint configuration.

Answer

The container must listen on port 8000 and use HTTP protocol.

Question 11

An ML team is using SageMaker Model Registry to manage model versions. After training a new model version, they register it with an 'Approved' status. The CI/CD pipeline automatically deploys the latest approved model to a staging endpoint. However, the pipeline fails with an error: 'Cannot deploy model because the model version is not approved.' The model version is clearly approved in the registry. What is the most likely cause?

Accepted Answer

The approval status was set on the model package group, not on the specific model version.. Option D is correct because in SageMaker Model Registry, approval is a property of a specific model version within a model package group, not of the model package group itself. The error indicates the pipeline is likely referencing the model package group ARN or a version that lacks explicit approval, even though the team believes the model is approved. The CI/CD pipeline must use the exact model version ARN that has the 'Approved' status to deploy successfully.

Answer

The pipeline is using the model package ARN instead of the model version ARN.

Answer

The model version is approved but the pipeline uses a different version that is still pending.

Answer

The SageMaker endpoint configuration does not have the necessary IAM permissions to read the registry.

Question 12

A company uses SageMaker Ground Truth to create a labeled dataset, then trains a model using SageMaker Training. They want to automate the pipeline so that whenever a labeling job is completed, it triggers the training job. Which architecture meets this requirement with minimal latency?

Accepted Answer

Use Amazon CloudWatch Events (EventBridge) to detect the completed labeling job and trigger a SageMaker Pipeline execution.. Option C is correct because Amazon EventBridge can natively capture SageMaker job state changes (e.g., `SageMaker Labeling Job State Change` to `Completed`) and directly trigger a SageMaker Pipeline execution. This event-driven approach eliminates polling overhead and provides the lowest latency by reacting immediately when the labeling job finishes.

Answer

Use AWS Step Functions to poll the labeling job status and then start training.

Answer

Configure an S3 event notification on the labeling job output bucket to trigger a Lambda function that starts training.

Answer

Set up a scheduled cron job in EventBridge to check for completed labeling jobs every hour and start training if found.

Question 13

A company deploys a model on SageMaker that serves predictions to a web application. The model's performance degrades over time due to data drift. The company wants to set up continuous monitoring. Which TWO actions should the company take to monitor and retrain the model effectively? (Choose TWO.)

Accepted Answer

Configure an Amazon EventBridge rule to start a retraining pipeline when the Model Monitor detects violations.. Option B is correct because Amazon EventBridge can be configured to trigger a retraining pipeline automatically when SageMaker Model Monitor detects data drift or other violations, enabling a closed-loop monitoring and retraining system. Option C is correct because SageMaker Model Monitor must first be enabled to capture inference data and run monitoring schedules, which is the prerequisite for detecting drift and triggering automated actions.

Answer

Manually review model performance monthly and retrain if necessary.

Answer

Use Amazon CloudWatch Logs Insights to query inference logs for anomalies.

Answer

Deploy the model on multiple endpoints with A/B testing to compare performance.

Question 14

A team is deploying a model using SageMaker Pipelines. They have defined a pipeline with steps: preprocessing, training, evaluation, and conditional registration. The evaluation step produces a JSON file with metrics. If accuracy > 0.9, the model is registered; else, the pipeline fails. Which TWO statements about this pipeline are correct? (Choose TWO.)

Accepted Answer

The evaluation step must output a JSON file in a specific format to be used by the condition step.. Option A is correct because the SageMaker Pipelines condition step expects the evaluation step to output a JSON file with a specific format, typically containing a metrics dictionary. The condition step then uses a property file to extract the accuracy value from that JSON, enabling the conditional logic to evaluate whether accuracy > 0.9.

Answer

The conditional step should be implemented as a separate Lambda function called from the pipeline.

Answer

The pipeline will automatically retry the training step if the condition fails.

Answer

The model registration step should be placed before the condition step to ensure the model is always registered.

Question 15

A company wants to deploy its trained model to edge devices such as cameras and IoT devices. The model must run efficiently with low latency and minimal memory footprint. Which THREE actions should the company take to prepare the model for edge deployment? (Choose THREE.)

Accepted Answer

Use SageMaker Edge Manager to package and manage the model on devices.. SageMaker Edge Manager is purpose-built to package, optimize, and manage machine learning models on edge devices. It provides model packaging, runtime monitoring, and over-the-air updates, ensuring the model runs efficiently with low latency and minimal memory footprint on resource-constrained hardware like cameras and IoT devices.

Answer

Increase the model's complexity to improve accuracy on edge devices.

Answer

Deploy the model directly as a SageMaker endpoint and have the edge devices call it over the internet.