MLA-C01 Deployment and Orchestration of ML Workflows Practice Test 9 — 15 Questions

Question 1

An MLOps team is designing a SageMaker Pipeline to automate model retraining. The pipeline must: (1) run training only if new training data is available, (2) register the model in SageMaker Model Registry only if evaluation metrics exceed a threshold, (3) deploy the approved model to a staging endpoint automatically. Which THREE steps should they include? (Choose THREE.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

ConditionStep to check if evaluation metrics exceed the threshold. Option E (ProcessingStep to check for new training data availability) is correct because a ProcessingStep can run a script or job that inspects the input data source and outputs a boolean/property indicating whether new data exists, which is exactly what requirement (1) needs before training runs. Option A (ConditionStep to check if evaluation metrics exceed the threshold) is correct because ConditionStep is the pipeline construct that evaluates a condition (e.g., a metric from a previous evaluation step against a threshold) and gates downstream steps, satisfying requirement (2). Option B (RegisterModel step to register the model in the Model Registry) is correct because RegisterModel is the dedicated SageMaker Pipelines step that creates a model package in the Model Registry, which is required to register the model once the metric condition passes. Option C (TuningStep) is not correct because hyperparameter optimization is not required by any of the stated requirements. Option D (TransformStep) is not correct because TransformStep runs batch transform inference jobs, not endpoint deployment; deploying to a staging endpoint would use a different mechanism such as a LambdaStep or a model deployment step, so it does not satisfy requirement (3).

Answer

TuningStep to perform hyperparameter optimization

Answer

TransformStep to deploy the model to a staging endpoint

Question 2

A company runs a real-time fraud detection model on a SageMaker endpoint. The model is updated weekly, and each update must be validated against live traffic without affecting existing predictions. The team wants to compare the new model's performance against the current model using a small percentage of incoming requests, while ensuring that the current model continues to serve the majority of traffic. Which SageMaker deployment strategy should they use?

Accepted Answer

A/B testing with production variants, where the new model variant receives a small portion of traffic.. SageMaker production variants enable A/B testing by allowing multiple models on a single endpoint with configurable traffic weights. This lets the team direct a small percentage of live traffic to the new model while the current model serves the rest, facilitating performance comparison without disrupting service. Other strategies either do not return predictions or switch all traffic at once.

Answer

Canary deployment, where the new model is deployed to a separate endpoint and traffic is gradually shifted using a load balancer.

Answer

Blue/green deployment, where traffic is shifted all at once from the old model to the new model after validation.

Answer

Shadow testing, where the new model receives a copy of live traffic but its predictions are not returned to the caller.

Question 3

A machine learning engineer is using SageMaker Pipelines to orchestrate a training workflow. The pipeline includes a processing step that outputs a dataset, which is then used by a training step. The engineer notices that the processing step runs every time the pipeline executes, even when the input data has not changed. The engineer wants to avoid re-running the processing step if the input data and code are unchanged, while ensuring that downstream steps still execute if the processing step is skipped. Which approach should the engineer take?

Accepted Answer

Enable caching on the processing step by setting the cache policy to CacheConfig with a time-to-live (TTL) and a cache key that includes the input data S3 URI and the processing script's S3 URI.. SageMaker Pipelines supports step caching, where the cache key is derived from the step's input artifacts and parameters. By configuring a cache policy with a TTL on the processing step and ensuring the cache key includes the input data URI and script URI, the step is skipped when these inputs are unchanged. The cached outputs are then used by downstream steps, maintaining pipeline integrity.

Answer

Use a condition step to check if the input data has changed, and if not, skip the processing step and directly pass the previous output to the training step.

Answer

Configure the processing step to use a spot instance, which will reduce cost but not affect whether the step runs.

Answer

Set the pipeline's execution mode to 'Reprocess' and manually skip the processing step by editing the pipeline definition before each run.

Question 4

A machine learning engineer has trained a scikit-learn model and saved it as model.joblib in Amazon S3. The engineer wants SageMaker to host the model for real-time inference without writing a custom container or inference script, because the model uses only standard predict behavior. Which deployment approach should the engineer use?

Accepted Answer

Deploy the model with the SageMaker Scikit-learn built-in framework container using the SageMaker Python SDK SKLearnModel class, pointing model_data to the S3 artifact.. Because the model is a standard scikit-learn artifact and no custom inference logic is required, the built-in Scikit-learn framework container is the correct fit. Passing the S3 model artifact to the SKLearnModel class lets SageMaker load the joblib file and use the container's default predict handler, producing a real-time endpoint without custom code or image maintenance.

Answer

Create a custom Docker image that installs scikit-learn, push it to Amazon ECR, and write an inference handler for the /invocations route.

Answer

Use SageMaker Batch Transform with the built-in Scikit-learn container and invoke the endpoint for each real-time request.

Answer

Upload model.joblib to SageMaker Model Registry and rely on automatic endpoint creation when the model package is approved.

Question 5

A machine learning engineer is deploying a real-time inference endpoint on Amazon SageMaker AI for a fraud detection model. The model must serve predictions with consistent latency under 50 ms and the team expects traffic to fluctuate unpredictably, with occasional bursts. The engineer wants to automatically adjust the number of instances based on actual workload while minimizing cost during idle periods. Which SageMaker AI feature should the engineer configure?

Accepted Answer

SageMaker AI automatic scaling with a target-tracking policy based on the SageMakerVariantInvocationsPerInstance metric. Automatic scaling with target-tracking on the invocations-per-instance metric is the correct approach because it continuously adjusts the number of instances to match actual traffic, ensuring latency targets are met while scaling down during idle periods to save cost. The other options either provide static recommendations or are designed for different inference patterns.

Answer

SageMaker AI multi-model endpoints with a shared inference container

Answer

SageMaker AI Inference Recommender to select the optimal instance type and count

Answer

SageMaker AI Asynchronous Inference with an auto-scaling policy on queue depth

Question 6

A machine learning team is deploying a model to a SageMaker endpoint and needs to implement A/B testing between two model versions. They want to split traffic 80/20 and monitor performance metrics for each variant. Which two actions should they take? (Choose two.)

Accepted Answer

Create a SageMaker endpoint configuration with two production variants, each specifying a different model and initial weight.. To perform A/B testing on SageMaker, you create an endpoint configuration with multiple production variants, each with a different model and weight to split traffic. Enabling data capture logs the requests and responses for each variant, allowing you to analyze performance metrics. Other options either do not provide native A/B testing or add unnecessary complexity.

Answer

Configure an Application Load Balancer to distribute traffic between two separate endpoints.

Answer

Use SageMaker Model Monitor to automatically compare the accuracy of the two variants.

Answer

Create two separate SageMaker endpoints and use AWS Lambda to route requests based on a random number.

Question 7

A machine learning engineer is deploying a model to a SageMaker endpoint that must handle occasional large payloads up to 1 GB. The inference time can take up to 10 minutes. The team wants to minimize cost and avoid idle compute. Which deployment option is most appropriate?

Accepted Answer

SageMaker Asynchronous Inference. Asynchronous Inference is designed for large payloads (up to 1 GB) and long inference times (up to 15 minutes). It queues requests and returns results via S3, and it can scale to zero when idle, reducing cost. Real-time and serverless inference have payload and timeout limits, and batch transform is for offline batch processing, not on-demand requests.

Answer

SageMaker real-time endpoint with automatic scaling

Answer

SageMaker Serverless Inference

Answer

SageMaker batch transform

Question 8

An ML engineer has a real-time SageMaker endpoint serving a fraud-detection model. The team wants to release a new model version to a small percentage of live traffic first, monitor CloudWatch metrics for accuracy regressions, and roll back quickly if performance degrades. They also want the production and candidate variants to share the same endpoint so latency comparisons are apples-to-apples. Which SageMaker deployment strategy should they use?

Accepted Answer

Configure the endpoint with two production variants (current and candidate) and set initial variant weights, using CloudWatch alarms to trigger rollback.. A single SageMaker endpoint can host multiple production variants, each with its own model artifact and a traffic weight that you adjust without redeploying infrastructure. This provides canary-style exposure, apples-to-apples latency on shared instances, and a fast rollback path when CloudWatch alarms fire. It is the native SageMaker mechanism for controlled model releases with live monitoring, matching every stated requirement.

Answer

Update the existing endpoint in place by replacing the production variant's model with the new model artifact.

Answer

Create a second endpoint with the new model and use Route 53 weighted routing to split traffic.

Answer

Deploy the new model as a production variant on the existing endpoint and configure a shadow variant with zero traffic weight.

Question 9

A machine learning engineer has trained a model using SageMaker and wants to deploy it to a real-time endpoint. The engineer needs to specify the model artifacts, the inference code, and the environment. Which SageMaker resource should the engineer create first?

Accepted Answer

A SageMaker model. The deployment sequence in SageMaker begins with creating a model, which specifies the model artifacts and the inference container. This model is then referenced in an endpoint configuration, which defines the deployment settings. Finally, the endpoint is created from that configuration. Therefore, the model is the first resource to create.

Answer

An endpoint configuration

Answer

A SageMaker pipeline

Answer

A SageMaker endpoint

Question 10

A company is deploying a large language model (LLM) to a SageMaker endpoint. They want to minimize inference latency and cost by using GPU acceleration and model parallelism. The model is too large to fit on a single GPU. Which SageMaker feature should they use?

Accepted Answer

SageMaker model parallelism library. The SageMaker model parallelism library is designed to partition large models across multiple GPUs, enabling inference for models that exceed a single GPU's memory. It supports various parallelism strategies and can reduce latency and cost. The other options are either for training, deprecated, or only provide recommendations, not the required model parallelism.

Answer

SageMaker Elastic Inference

Answer

SageMaker distributed data parallel library

Answer

SageMaker Inference Recommender

Question 11

A team runs a SageMaker Pipeline that trains a model and registers it in the Model Registry. Compliance requires that the pipeline run automatically every time new labeled data lands in S3, and that each run record the exact S3 data prefix, the training image URI, and the git commit hash as lineage metadata. The engineer wants the least operational overhead. Which approach meets these requirements?

Accepted Answer

Configure an S3 event notification to invoke a Lambda function that calls StartPipelineExecution with the data prefix and commit hash passed as pipeline parameters.. S3 event notifications driving a Lambda that invokes StartPipelineExecution provides immediate, event-driven execution with parameter overrides for the data prefix and commit hash. Those parameters propagate into pipeline steps and are captured as lineage metadata in the Model Registry, satisfying compliance. It avoids polling, scheduling waste, and manual gates, giving the lowest operational overhead of the listed designs.

Answer

Enable SageMaker Data Wrangler scheduled jobs to export data and manually start the pipeline after reviewing the export in SageMaker Studio.

Answer

Create an EventBridge scheduled rule that runs the pipeline every 15 minutes and relies on the pipeline's caching to skip runs when data is unchanged.

Answer

Use AWS Step Functions with a Wait state polling S3 every minute, then call StartPipelineExecution once new objects are detected.

Question 12

A data science team is using AWS Step Functions to orchestrate a machine learning workflow that includes a SageMaker training job followed by a model deployment. They want to ensure that if the training job fails, the workflow retries up to three times with exponential backoff before sending a notification to an Amazon SNS topic. Which Step Functions feature should they use to implement this?

Accepted Answer

Retry and Catch fields on the training task state, with a retry policy specifying MaxAttempts and BackoffRate, and a Catch field that transitions to an SNS publish state.. Step Functions' Retry and Catch fields on a state provide built-in error handling. By configuring Retry with MaxAttempts and BackoffRate, the training task will be retried with exponential backoff. The Catch field can then route to an SNS publish state after all retries fail, ensuring notification only on persistent failure.

Answer

A Map state that iterates over a list of retry attempts, invoking the training job each time until it succeeds.

Answer

A Choice state that checks the training job status and loops back to the training state if it failed, with a counter to limit attempts.

Answer

A Parallel state that runs the training job and an SNS notification simultaneously, with a retry policy on the training job.

Question 13

A team is building a SageMaker Pipeline that trains a model and then registers it in the SageMaker Model Registry. They want the pipeline to automatically deploy the model to a real-time endpoint only after a human approves the model package. Which two actions should the team take to implement this approval-gated deployment? (Choose two.)

Accepted Answer

Add a RegisterModel step to the pipeline that creates a model package with a PendingManualApproval status.. Manual approval gating relies on the model package lifecycle: RegisterModel creates a package in PendingManualApproval, and a reviewer later approves it. Because the pipeline execution does not pause for review, an EventBridge rule watching for the Approved state change is what triggers the downstream deployment automation, keeping humans in the loop.

Answer

Set the model package status to Approved directly in the RegisterModel step configuration so the endpoint deploys without human intervention.

Answer

Use a SageMaker Clarify processing step to approve the model package when bias metrics fall within thresholds.

Answer

Add a ConditionStep to the pipeline that checks the model package status immediately after RegisterModel and branches to a deployment step.

Question 14

A machine learning team runs a SageMaker AI Pipeline that trains a model and registers it in the SageMaker AI Model Registry. A separate deployment process must promote the model to production only after a human reviewer approves the model version. The team wants to automate the promotion so that approval in the Model Registry triggers the deployment without manual intervention. Which combination of steps should the engineer implement?

Accepted Answer

Configure an Amazon EventBridge rule for the SageMaker AI Model Registry model version state change to Approved, and use it to start a deployment workflow.. Model approval in the SageMaker AI Model Registry changes the model version state and emits an event. Capturing that event with Amazon EventBridge and routing it to a deployment workflow links the human approval gate to automated promotion, which is exactly the required behavior.

Answer

Use a SageMaker AI Projects template that automatically deploys every model version as soon as it is registered.

Answer

Schedule an AWS Lambda function every minute to call DescribeModelPackage and deploy any approved model version.

Answer

Add a ConditionStep to the existing pipeline that checks the model version status and deploys the model.

Question 15

A media company runs a real-time recommendation model on a SageMaker endpoint. Traffic triples every evening between 18:00 and 22:00 and drops to near zero overnight, and the team wants to cut costs without underserving evening users. They want the endpoint to scale out automatically as invocations rise and scale back in when traffic falls. Which solution should they implement?

Accepted Answer

Enable automatic scaling on the endpoint with a target-tracking scaling policy based on the SageMakerVariantInvocationsPerInstance metric.. Target-tracking automatic scaling based on SageMakerVariantInvocationsPerInstance is the native mechanism for elastic real-time endpoints. It registers the variant as a scalable target with Application Auto Scaling, then adds or removes instances to hold the metric near the target. This directly matches a recurring evening peak followed by an overnight trough while keeping latency low, unlike fixed serverless concurrency or static over-provisioning.

Answer

Create a second endpoint and use an Application Load Balancer to route evening traffic between the two endpoints.

Answer

Deploy the model to a SageMaker Serverless Inference endpoint and set the maximum concurrency to a fixed value equal to the evening peak.

Answer

Increase the instance count of the endpoint to the evening peak permanently and rely on Savings Plans to offset the extra cost.