MLA-C01 Deployment and Orchestration of ML Workflows Practice Test 5 — 15 Questions

Question 1

A team built a SageMaker Pipeline that includes a training step and a model evaluation step. They want to automatically register a model in SageMaker Model Registry only if the evaluation metric (accuracy) exceeds 0.9. Which pipeline step should be used to implement this conditional logic?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Condition step. The Condition step in SageMaker Pipelines allows you to add conditional branching logic, such as evaluating a metric and proceeding only if a condition is met. In this scenario, you would use a ConditionStep to check if the accuracy metric from the evaluation step exceeds 0.9, and then conditionally execute a RegisterModel step to register the model in SageMaker Model Registry. A common misconception is that the RegisterModel step itself can conditionally register a model based on metrics, but in SageMaker Pipelines, conditional logic must be implemented explicitly with a ConditionStep.

Answer

RegisterModel step

Answer

Processing step

Answer

Transform step

Question 2

A data science team uses SageMaker Pipelines to orchestrate their ML workflow. They noticed that even when source data hasn't changed, the pipeline re-runs all steps, wasting compute time. What should they enable to avoid redundant runs?

Accepted Answer

Enable pipeline caching by setting the CacheConfig property for each step. SageMaker Pipelines supports step caching via the `CacheConfig` property. When enabled, the pipeline checks if the step's inputs (including source data, parameters, and code) have changed since the last successful run. If no changes are detected, the step is skipped and the previous output is reused, eliminating redundant compute.

Answer

Configure the pipeline to run on a schedule instead of on-demand

Answer

Use the Parameter step to pass previous execution ID

Answer

Use Lambda step to check data changes before running

Question 3

A company wants to update an existing SageMaker real-time endpoint to serve a new model version. They need to route a small percentage of traffic to the new version initially and monitor for errors before switching fully. Which deployment pattern supports this?

Accepted Answer

Canary deployment with weighted production variants. SageMaker real-time endpoints support canary deployments by configuring multiple production variants with weighted traffic distribution. You can assign a small weight (e.g., 5%) to the new model version variant and 95% to the existing one, then monitor CloudWatch metrics for errors before shifting all traffic to the new variant. This matches the requirement for a gradual, monitored rollout.

Answer

Shadow testing

Answer

A/B testing with traffic splitting

Answer

Blue/green deployment

Question 4

A company deploys a large NLP model on a SageMaker real-time endpoint using an ml.p3.2xlarge instance. To reduce inference cost without sacrificing throughput, they want to compile the model for their target hardware. Which service should they use?

Accepted Answer

SageMaker Neo. SageMaker Neo is the correct service because it compiles trained machine learning models into an optimized binary for a specific target hardware (e.g., ml.p3.2xlarge with NVIDIA GPUs). This reduces inference latency and cost by applying hardware-specific optimizations such as kernel fusion and memory layout tuning, while preserving the original model's throughput. The compilation process uses Apache TVM under the hood to generate efficient code for the target instance type.

Answer

Triton Inference Server on SageMaker

Answer

Amazon Elastic Inference

Answer

SageMaker Inference Recommender

Question 5

A team uses SageMaker real-time endpoints for inference. They want to deploy a new model version and compare its performance with the current version under live traffic without affecting user experience. Which method should they use?

Accepted Answer

Shadow testing with SageMaker. Shadow testing with SageMaker deploys the new model alongside the production model and mirrors live traffic to it without returning its responses to users. This lets the team compare performance under real traffic with zero impact on user experience.

Answer

A/B testing with production variant traffic splitting

Answer

Batch transform on a holdout test set

Answer

Blue/green deployment

Question 6

A company wants to deploy a machine learning model using infrastructure as code to ensure reproducibility. They need to define the SageMaker Studio domain, user profiles, and the endpoint configuration. Which tool should they use?

Accepted Answer

AWS CloudFormation or AWS CDK. AWS CloudFormation and AWS CDK are infrastructure-as-code (IaC) tools that allow you to define, provision, and manage AWS resources declaratively. For this use case, they can model the entire SageMaker Studio domain, user profiles, and endpoint configuration in templates or code, ensuring reproducibility and version control. This aligns directly with the requirement to deploy ML infrastructure as code.

Answer

SageMaker Pipelines

Answer

AWS Step Functions

Answer

SageMaker Studio

Question 7

A company uses SageMaker Pipelines to automate their ML workflow. They need to add model versioning and approval workflow. Which THREE steps should they include in their pipeline to achieve this? (Choose THREE.)

Accepted Answer

RegisterModel step. Option A (RegisterModel step) is correct because it creates a model package in the SageMaker Model Registry, which is the core mechanism for versioning models and tracking their approval status. Option C (Condition step) is correct because it enables the pipeline to branch based on evaluation results or approval status, allowing the workflow to proceed only when the model meets criteria or is approved. Option D (Processing step for evaluation) is correct because it runs the evaluation logic that produces the metrics used by the Condition step to decide whether the model should be registered or approved. Option B (Training step) is not one of the three required steps here since training alone does not provide versioning or approval; it is typically a prerequisite but not the mechanism for versioning/approval. Option E (Transform step) is not correct because batch transform is for inference, not for model versioning or approval workflow.

Answer

Training step

Answer

Transform step

Question 8

A machine learning team has a model that needs to serve predictions with very low latency (under 10 ms) for a real-time web application. The model is a small ensemble of three neural networks that fits in memory. Which SageMaker inference option is MOST appropriate?

Accepted Answer

SageMaker real-time endpoint. SageMaker real-time endpoints are designed for low-latency, synchronous inference, making them the best fit for a model that must serve predictions in under 10 ms. Since the ensemble of three neural networks fits in memory, a real-time endpoint can keep the model loaded and respond to each request with minimal overhead, typically using HTTPS and the SageMaker InvokeEndpoint API.

Answer

SageMaker batch transform

Answer

SageMaker asynchronous inference

Answer

SageMaker serverless inference

Question 9

A data science team wants to host 50 different models for a recommendation engine. Each model is small (under 100 MB) and traffic patterns are unpredictable. They need to minimize cost and operational overhead. Which approach should they take?

Accepted Answer

Use a single multi-model endpoint (MME). A single multi-model endpoint (MME) allows hosting multiple models (up to thousands) on the same endpoint, sharing the underlying compute instance. This minimizes cost and operational overhead for small models (under 100 MB) with unpredictable traffic, as the endpoint dynamically loads and unloads models from Amazon S3 into memory based on incoming requests, eliminating the need for separate endpoints or idle compute.

Answer

Deploy each model to its own real-time endpoint

Answer

Use SageMaker serverless inference for each model

Answer

Use a single multi-container endpoint

Question 10

A machine learning engineer needs to deploy a new version of a model gradually, initially sending 5% of traffic to the new version and 95% to the current version, while monitoring for errors. Which deployment pattern should they use?

Accepted Answer

Canary deployment. Canary deployment is the correct pattern because it allows the ML engineer to route a small percentage of traffic (e.g., 5%) to the new model version while keeping the majority (95%) on the current version. This enables gradual rollout with real-time monitoring for errors, and if issues are detected, traffic can be instantly shifted back to the stable version.

Answer

Blue/green deployment

Answer

Shadow testing

Answer

Rolling deployment

Question 11

A company is using SageMaker Pipelines to orchestrate their ML workflow. They have a Condition step that checks if a model's accuracy exceeds 0.9. If true, they want to register the model in the model registry; otherwise, they want to run a retraining step. Which step type should they use for the decision?

Accepted Answer

Condition step. A Condition step in SageMaker Pipelines evaluates a boolean expression against a property (such as a model accuracy metric from a preceding Evaluation step) and branches the pipeline into a true or false path. This is exactly the construct needed to register the model when accuracy > 0.9 and otherwise trigger retraining.

Answer

Transform step

Answer

Processing step

Answer

Tuning step

Question 12

An organization wants to ensure that only approved model versions can be deployed to production. They use the SageMaker Model Registry to track model versions. How can they enforce that only approved models are deployed?

Accepted Answer

Use IAM policies to restrict deployment to only Approved model versions. AWS IAM policies can be used to conditionally restrict SageMaker API actions (e.g., CreateEndpointConfig, CreateModel) based on the model version's approval status. By evaluating the `sagemaker:ModelPackageApprovalStatus` condition key in an IAM policy, you can enforce that only model versions with an `Approved` status can be deployed, providing a native, automated, and auditable enforcement mechanism without manual intervention or external dependencies.

Answer

Manually review each model before deployment

Answer

Use SageMaker Model Monitor to check model quality after deployment

Answer

Store model metadata in a DynamoDB table and check it before deployment

Question 13

A company uses SageMaker Pipelines to automate their ML workflow. They notice that the pipeline reruns all steps even when the input data has not changed. Which feature should they enable to avoid unnecessary recomputation?

Accepted Answer

Enable pipeline caching. Pipeline caching in SageMaker Pipelines automatically reuses the output of a step if its inputs (including parameters, data, and code) have not changed since the last successful execution. This avoids recomputation by comparing a hash of the step's dependencies against previous runs, making it the correct feature to prevent unnecessary reruns when input data remains identical.

Answer

Use a Lambda step to check input changes

Answer

Use a Conditional step to skip steps

Answer

Set the pipeline execution mode to 'Parallel'

Question 14

An ML engineer wants to use MLflow on SageMaker to track experiments and log metrics. They have set up MLflow on an EC2 instance. How can they best integrate MLflow tracking with SageMaker training jobs?

Accepted Answer

Set the MLFLOW_TRACKING_URI environment variable in the training job and use the mlflow library in the training script. The ML engineer can set the `MLFLOW_TRACKING_URI` environment variable in the SageMaker training job definition and use the `mlflow` library inside the training script to log parameters, metrics, and artifacts directly to the MLflow tracking server running on the EC2 instance. This approach allows the training job to communicate with the external MLflow server over HTTP/HTTPS without requiring any additional SageMaker integrations.

Answer

Install MLflow on the SageMaker notebook instance only

Answer

Use the SageMaker Experiments integration with MLflow

Answer

Use SageMaker Processing to run MLflow after training

Question 15

A company needs to update a model in production without any downtime. They currently have a single real-time endpoint serving traffic. Which approach allows them to deploy a new model version and switch traffic gradually while being able to roll back quickly?

Accepted Answer

Use a canary deployment by creating a new production variant with the new model and shifting traffic incrementally. A canary deployment creates a new production variant on the existing endpoint with the new model and shifts a small percentage of traffic to it, allowing gradual validation and quick rollback by shifting traffic back to the original variant. This avoids downtime and provides controlled risk.

Answer

Use a multi-model endpoint and replace the model file

Answer

Stop the endpoint, update the model, and restart the endpoint

Answer

Update the existing endpoint's model directly using UpdateEndpoint