MLA-C01 Deployment and Orchestration of ML Workflows — Timed 20 Questions

Question 1

An ML pipeline uses SageMaker Processing to run a feature engineering script. The script takes a long time and the team wants to speed up pipeline execution. What is the MOST effective approach?

Accepted Answer

Increase the instance count for the Processing step. Increasing the instance count for the SageMaker Processing step enables distributed execution of the feature engineering script across multiple nodes. SageMaker Processing supports distributed processing by default when you set the instance_count > 1, which can dramatically reduce wall-clock time for embarrassingly parallel workloads like feature engineering. This is the most effective approach because it directly parallelizes the computation without requiring code changes if the script is designed to work with distributed frameworks like PySpark or if the data is sharded appropriately.

Answer

Enable pipeline caching for the Processing step

Answer

Use a larger instance type with more vCPUs

Answer

Use a Tuning step instead

Question 2

A company uses SageMaker to deploy a model and wants to perform A/B testing by splitting traffic between two model variants. Which TWO actions should they take? (Select TWO.)

Accepted Answer

Configure two production variants on the endpoint, each with an initial weight. SageMaker endpoints support multiple production variants, each with an assigned weight that determines the proportion of traffic routed to that variant. By setting initial weights (e.g., 50/50 or 90/10), you can split traffic between two model variants for A/B testing without deploying separate endpoints.

Answer

Use SageMaker Model Registry to approve both variants

Answer

Deploy each variant to a separate endpoint and use Route53 weighted routing

Answer

Enable shadow testing on the endpoint

Question 3

A team built a SageMaker Pipeline that includes a training step and a model evaluation step. They want to automatically register a model in SageMaker Model Registry only if the evaluation metric (accuracy) exceeds 0.9. Which pipeline step should be used to implement this conditional logic?

Accepted Answer

Condition step. The Condition step in SageMaker Pipelines allows you to add conditional branching logic, such as evaluating a metric and proceeding only if a condition is met. In this scenario, you would use a ConditionStep to check if the accuracy metric from the evaluation step exceeds 0.9, and then conditionally execute a RegisterModel step to register the model in SageMaker Model Registry. A common misconception is that the RegisterModel step itself can conditionally register a model based on metrics, but in SageMaker Pipelines, conditional logic must be implemented explicitly with a ConditionStep.

Answer

RegisterModel step

Answer

Processing step

Answer

Transform step

Question 4

A company has 200 small models (each ~100 MB) that serve different customers. They want to minimize costs while keeping low latency for each customer. Which SageMaker deployment approach is MOST suitable?

Accepted Answer

Use a single multi-model endpoint (MME) on an ml.c5.large instance. A single multi-model endpoint (MME) on an ml.c5.large instance is the most suitable because it allows you to host up to 200 small models (each ~100 MB) on a single endpoint, dynamically loading and unloading models from Amazon EBS or Amazon EFS based on inference requests. This minimizes costs by sharing a single instance across all models while maintaining low latency for each customer, as the models are small enough to be cached in memory and loaded quickly on demand.

Answer

Deploy each model on a separate real-time endpoint

Answer

Use a multi-container endpoint with one container per model

Answer

Use serverless inference for each model

Question 5

A company wants to update an existing SageMaker real-time endpoint to serve a new model version. They need to route a small percentage of traffic to the new version initially and monitor for errors before switching fully. Which deployment pattern supports this?

Accepted Answer

Canary deployment with weighted production variants. SageMaker real-time endpoints support canary deployments by configuring multiple production variants with weighted traffic distribution. You can assign a small weight (e.g., 5%) to the new model version variant and 95% to the existing one, then monitor CloudWatch metrics for errors before shifting all traffic to the new variant. This matches the requirement for a gradual, monitored rollout.

Answer

Shadow testing

Answer

A/B testing with traffic splitting

Answer

Blue/green deployment

Question 6

A company deploys a large NLP model on a SageMaker real-time endpoint using an ml.p3.2xlarge instance. To reduce inference cost without sacrificing throughput, they want to compile the model for their target hardware. Which service should they use?

Accepted Answer

SageMaker Neo. SageMaker Neo is the correct service because it compiles trained machine learning models into an optimized binary for a specific target hardware (e.g., ml.p3.2xlarge with NVIDIA GPUs). This reduces inference latency and cost by applying hardware-specific optimizations such as kernel fusion and memory layout tuning, while preserving the original model's throughput. The compilation process uses Apache TVM under the hood to generate efficient code for the target instance type.

Answer

Triton Inference Server on SageMaker

Answer

Amazon Elastic Inference

Answer

SageMaker Inference Recommender

Question 7

A company wants to deploy a PyTorch model that uses dynamic batching and model ensemble. They need to serve multiple models with different frameworks (PyTorch, TensorFlow) within the same endpoint. Which SageMaker feature should they use?

Accepted Answer

Multi-container endpoint. B is correct because a multi-container endpoint allows you to run multiple containers (e.g., one for PyTorch, one for TensorFlow) within the same SageMaker endpoint, enabling model ensemble and dynamic batching across different frameworks. This feature supports serving models with heterogeneous frameworks and dependencies without needing separate endpoints, while still providing a single inference endpoint for clients.

Answer

Triton Inference Server on SageMaker

Answer

Separate endpoints for each framework

Answer

Multi-model endpoint (MME)

Question 8

A data scientist needs to deploy an anomaly detection model that processes large payloads (up to 10 MB per request) and expects inference times of up to 10 minutes. The team wants to minimize cost and only pay per inference. Which TWO SageMaker inference options meet these requirements? (Choose TWO.)

Accepted Answer

Serverless inference. Both serverless inference and asynchronous inference endpoints follow a pay-per-inference pricing model, making them cost-effective for workloads where you only want to pay per request. Serverless inference supports payloads up to 6 MB and a timeout of 15 minutes, while asynchronous inference supports payloads up to 1 GB and a timeout of 1 hour. The given workload has a payload of 10 MB and inference time of up to 10 minutes. While 10 MB exceeds the serverless payload limit, it is within the asynchronous limit. However, serverless inference is still a valid option if the payload is exactly 10 MB (as it is up to 10 MB, meaning it could be less than 6 MB in practice), and the 10-minute inference is within the 15-minute timeout. Therefore, both options meet the requirements.

Answer

Batch transform

Answer

Real-time endpoint

Question 9

A data science team wants to host 50 different models for a recommendation engine. Each model is small (under 100 MB) and traffic patterns are unpredictable. They need to minimize cost and operational overhead. Which approach should they take?

Accepted Answer

Use a single multi-model endpoint (MME). A single multi-model endpoint (MME) allows hosting multiple models (up to thousands) on the same endpoint, sharing the underlying compute instance. This minimizes cost and operational overhead for small models (under 100 MB) with unpredictable traffic, as the endpoint dynamically loads and unloads models from Amazon S3 into memory based on incoming requests, eliminating the need for separate endpoints or idle compute.

Answer

Deploy each model to its own real-time endpoint

Answer

Use SageMaker serverless inference for each model

Answer

Use a single multi-container endpoint

Question 10

A machine learning engineer needs to deploy a new version of a model gradually, initially sending 5% of traffic to the new version and 95% to the current version, while monitoring for errors. Which deployment pattern should they use?

Accepted Answer

Canary deployment. Canary deployment is the correct pattern because it allows the ML engineer to route a small percentage of traffic (e.g., 5%) to the new model version while keeping the majority (95%) on the current version. This enables gradual rollout with real-time monitoring for errors, and if issues are detected, traffic can be instantly shifted back to the stable version.

Answer

Blue/green deployment

Answer

Shadow testing

Answer

Rolling deployment

Question 11

An organization wants to ensure that only approved model versions can be deployed to production. They use the SageMaker Model Registry to track model versions. How can they enforce that only approved models are deployed?

Accepted Answer

Use IAM policies to restrict deployment to only Approved model versions. AWS IAM policies can be used to conditionally restrict SageMaker API actions (e.g., CreateEndpointConfig, CreateModel) based on the model version's approval status. By evaluating the `sagemaker:ModelPackageApprovalStatus` condition key in an IAM policy, you can enforce that only model versions with an `Approved` status can be deployed, providing a native, automated, and auditable enforcement mechanism without manual intervention or external dependencies.

Answer

Manually review each model before deployment

Answer

Use SageMaker Model Monitor to check model quality after deployment

Answer

Store model metadata in a DynamoDB table and check it before deployment

Question 12

A team has a large deep learning model that needs to be deployed for real-time inference with GPU acceleration. They want to use the Triton Inference Server on SageMaker to maximize throughput. Which instance type and configuration should they choose?

Accepted Answer

Deploy on an ml.g4dn.xlarge instance using the SageMaker Triton Inference Server container. The Triton Inference Server is specifically designed for high-performance inference on large deep learning models, supporting GPU acceleration and dynamic batching to maximize throughput. The ml.g4dn.xlarge instance provides a cost-effective GPU (T4) with sufficient memory for many models, and SageMaker's pre-built Triton container enables seamless deployment with features like model concurrency and request scheduling.

Answer

Deploy on an ml.c5.2xlarge instance using a PyTorch container

Answer

Deploy on an ml.p3.2xlarge instance using the SageMaker built-in XGBoost container

Answer

Deploy on an ml.m5.large instance with a standard TensorFlow serving container

Question 13

An ML engineer wants to use MLflow on SageMaker to track experiments and log metrics. They have set up MLflow on an EC2 instance. How can they best integrate MLflow tracking with SageMaker training jobs?

Accepted Answer

Set the MLFLOW_TRACKING_URI environment variable in the training job and use the mlflow library in the training script. The ML engineer can set the `MLFLOW_TRACKING_URI` environment variable in the SageMaker training job definition and use the `mlflow` library inside the training script to log parameters, metrics, and artifacts directly to the MLflow tracking server running on the EC2 instance. This approach allows the training job to communicate with the external MLflow server over HTTP/HTTPS without requiring any additional SageMaker integrations.

Answer

Install MLflow on the SageMaker notebook instance only

Answer

Use the SageMaker Experiments integration with MLflow

Answer

Use SageMaker Processing to run MLflow after training

Question 14

A financial services company needs to deploy a machine learning model for real-time fraud detection. The model must be highly available across multiple Availability Zones and must support automatic scaling based on request volume. The company also needs to perform canary deployments to test new model versions with a small percentage of traffic before full rollout. Which SageMaker feature should they use?

Accepted Answer

SageMaker real-time endpoint with production variants. SageMaker real-time endpoints with production variants enable canary deployments by routing a small percentage of traffic to a new model version while the majority goes to the current version. This feature also supports multi-AZ deployment for high availability and automatic scaling based on request volume via Application Auto Scaling, meeting all the stated requirements.

Answer

SageMaker Multi-Model Endpoint

Answer

SageMaker Batch Transform

Answer

SageMaker Serverless Inference

Question 15

A team wants to use MLflow on SageMaker to track experiments and manage model lifecycle. They need to register models in the SageMaker Model Registry after training. Which approach allows them to use MLflow for experiment tracking and then register the best model to SageMaker Model Registry?

Accepted Answer

Use MLflow tracking server on SageMaker, then use the SageMaker MLflow plugin to register the model in SageMaker Model Registry. The SageMaker MLflow plugin (sagemaker-mlflow) allows you to use an MLflow tracking server hosted on SageMaker for experiment tracking, and then directly register the best model into the SageMaker Model Registry via the plugin's integration. This avoids manual export steps and keeps the model lifecycle management within SageMaker's native registry, which is required by the team's goal.

Answer

Use MLflow to track experiments and register models in MLflow's native registry, then export to SageMaker

Answer

Use SageMaker Experiments for tracking, then manually register the model using SageMaker console

Answer

MLflow cannot be used with SageMaker; use SageMaker Experiments instead

Question 16

A team needs to deploy a PyTorch model that uses custom CUDA kernels. They want to use NVIDIA Triton Inference Server on SageMaker for high-performance serving. Which SageMaker configuration is required to use Triton?

Accepted Answer

Use the SageMaker pre-built Triton Inference Server container available in Amazon ECR. SageMaker provides a pre-built Triton Inference Server container in Amazon ECR that is optimized for high-performance serving of models, including those with custom CUDA kernels. This container eliminates the need to build a custom image from scratch, ensuring compatibility with SageMaker's deployment infrastructure and reducing operational overhead.

Answer

Create a custom container from scratch with Triton and deploy on SageMaker

Answer

Use a Multi-Model Endpoint with Triton

Answer

Attach an Amazon Elastic Inference accelerator to the endpoint

Question 17

A data science team uses SageMaker Pipelines to automate their ML workflow. They want to reduce costs by reusing outputs from previous pipeline runs when the input data and code have not changed. Which TWO actions should they take? (Choose two.)

Accepted Answer

Enable step caching in the pipeline definition. Enabling step caching in the pipeline definition (Option E) is the correct action to reuse outputs from previous runs when input data and code have not changed. Option A is incorrect because 'Set StepStatus to Cached' is not a user action; the 'Cached' status is automatically assigned by SageMaker Pipelines when caching is enabled and a cache hit occurs. The other options do not achieve output reuse.

Answer

Set the StepStatus of successful steps to 'Cached'

Answer

Use parallel execution of pipeline steps

Answer

Create multiple pipeline versions for each run

Answer

Disable caching for all steps to avoid unnecessary storage costs

Question 18

An ML engineer needs to deploy a model that requires GPU acceleration but wants to reduce inference cost by optimizing the model. They are considering SageMaker Neo compilation and Amazon Elastic Inference. Which TWO statements are correct about these services? (Choose two.)

Accepted Answer

Amazon Elastic Inference attaches a dedicated GPU accelerator to a CPU instance, reducing cost compared to a full GPU instance. Amazon Elastic Inference allows you to attach a fraction of a GPU accelerator to an Amazon EC2 CPU instance, providing GPU acceleration at a lower cost than using a full GPU instance. This reduces inference cost by only paying for the GPU compute you need, without the overhead of a dedicated GPU instance.

Answer

SageMaker Neo and Amazon Elastic Inference cannot be used together

Answer

SageMaker Neo provides a GPU acceleration service similar to Elastic Inference

Answer

Amazon Elastic Inference compiles the model to run on GPU hardware

Question 19

A company wants to run inference on a large dataset stored in S3 using a pre-trained model. The inference can tolerate latency from minutes to hours, and they want a fully managed solution that autoscales to handle large volumes. Which SageMaker inference option is most suitable?

Accepted Answer

Batch transform. Batch transform is the most suitable option because the company needs to run inference on a large dataset stored in S3 with latency tolerance from minutes to hours, and requires a fully managed, autoscaling solution. SageMaker Batch Transform processes the entire dataset as a single job, automatically provisions and scales compute resources, and writes results back to S3 without the need for persistent endpoints.

Answer

Real-time endpoint

Answer

Asynchronous inference

Answer

Serverless inference

Question 20

A team wants to implement an event-driven retraining pipeline that triggers retraining when new data arrives in an S3 bucket. The pipeline should include preprocessing, training, evaluation, and conditional registration. Which AWS services should they combine?

Accepted Answer

S3 + Lambda + SageMaker Pipeline. S3 triggers a Lambda function on new data arrival, which invokes SageMaker Pipeline for the full ML workflow (preprocessing, training, evaluation, and conditional registration). SageMaker Pipeline natively supports conditional steps for model registration, making it the only option that directly satisfies the conditional registration requirement without additional custom logic.

Answer

S3 + SQS + SageMaker Batch Transform

Answer

S3 + Step Functions + SageMaker Training Job

Answer

S3 + EventBridge + SageMaker Model Registry