Deployment and Orchestration of ML Workflows practice questions

Enable pipeline caching for the Processing step

Why wrong: Caching skips the step if inputs unchanged, but does not speed up execution when inputs change.

Use a larger instance type with more vCPUs

Why wrong: While a larger instance helps, parallelization with multiple instances is more effective for scalable speedup.

Use a Tuning step instead

Why wrong: Tuning is for hyperparameter optimization, not feature engineering.

Use SageMaker Model Registry to approve both variants

Why wrong: Model Registry is for version management, not traffic splitting.

Deploy each variant to a separate endpoint and use Route53 weighted routing

Why wrong: Not the recommended SageMaker approach; SageMaker supports native traffic splitting.

Enable shadow testing on the endpoint

Why wrong: Shadow testing duplicates traffic but does not serve real traffic from the shadow.

RegisterModel step

Why wrong: RegisterModel step registers a model but does not provide conditional logic; it must be used within a branch.

Processing step

Why wrong: Processing steps run data processing or evaluation jobs but do not provide conditional branching natively.

Transform step

Why wrong: Transform steps run batch transform jobs and cannot branch based on conditions.

Deploy each model on a separate real-time endpoint

Why wrong: Separate endpoints for 200 models would be expensive due to many idle instances.

Use a multi-container endpoint with one container per model

Why wrong: Multi-container endpoints are for running different containers per request, not for hosting many models; they also have a container limit.

Use serverless inference for each model

Why wrong: Serverless has a per-model invoke limit and may incur cold starts; for 200 models, managing separate serverless endpoints is complex and not cost-optimal.

Shadow testing

Why wrong: Shadow testing sends duplicate traffic to a shadow variant but does not serve live traffic; it's for testing without impacting users.

A/B testing with traffic splitting

Why wrong: A/B testing is for comparing model performance; canary deployment is the specific pattern for gradual rollout.

Blue/green deployment

Why wrong: Blue/green switches all traffic at once; it does not support gradual traffic shifting.

Triton Inference Server on SageMaker

Why wrong: Triton is a serving framework that can support multiple frameworks and dynamic batching, but it does not compile models.

Amazon Elastic Inference

Why wrong: Elastic Inference attaches GPU acceleration to CPU instances but does not compile models; it's a hardware add-on.

SageMaker Inference Recommender

Why wrong: Inference Recommender helps choose the right instance type and configuration but does not compile models.

Triton Inference Server on SageMaker

Why wrong: Triton can serve multiple models from different frameworks within one container, but the question asks for multiple containers.

Separate endpoints for each framework

Why wrong: Separate endpoints increase cost and complexity; multi-container endpoints unify them.

Multi-model endpoint (MME)

Why wrong: MME hosts multiple models from the same framework in a single container; it does not support different frameworks natively.

Batch transform

Why wrong: Batch transform processes entire datasets offline and charges per instance hour, not per inference. It does not meet the pay-per-inference requirement.

Real-time endpoint

Why wrong: Real-time endpoints are always running and charge for the endpoint uptime, not per inference. They also have a payload limit of 5 MB and a 60-second timeout, making them unsuitable.

Deploy each model to its own real-time endpoint

Why wrong: 50 endpoints would be costly and operationally heavy, and each endpoint would have separate auto-scaling policies.

Use SageMaker serverless inference for each model

Why wrong: 50 serverless endpoints would have high management overhead and cold start issues for unpredictable traffic.

Use a single multi-container endpoint

Why wrong: Multi-container endpoints are for running different containers together for a single model, not for hosting many separate models.

Blue/green deployment

Why wrong: Blue/green deployment switches all traffic at once, not gradually.

Shadow testing

Why wrong: Shadow testing sends duplicate traffic to both versions but the new version's responses are not used; it is not a gradual rollout.

Rolling deployment

Why wrong: Rolling deployment is not a standard SageMaker term; canary is the correct term for gradual traffic shift.

Manually review each model before deployment

Why wrong: Manual review is error-prone and not scalable; it does not enforce the policy in code.

Use SageMaker Model Monitor to check model quality after deployment

Why wrong: Model Monitor monitors deployed models, but does not prevent deployment of unapproved models.

Store model metadata in a DynamoDB table and check it before deployment

Why wrong: Storing metadata in DynamoDB requires a custom deployment pipeline to query the table and enforce approval status, whereas SageMaker Model Registry natively integrates with Model Registry approval statuses and SageMaker Pipelines to gate deployments. This approach is tempting because DynamoDB is a flexible store for arbitrary model metadata, and it would be correct if the organisation needed to enforce custom approval logic not supported by the registry’s built-in approval workflow.

Deploy on an ml.c5.2xlarge instance using a PyTorch container

Why wrong: ml.c5 instances are CPU-only; Triton can run on CPU but GPU acceleration is desired.

Deploy on an ml.p3.2xlarge instance using the SageMaker built-in XGBoost container

Why wrong: The XGBoost container does not support Triton; you need the Triton container.

Deploy on an ml.m5.large instance with a standard TensorFlow serving container

Why wrong: ml.m5 instances do not have GPU support, and the standard TensorFlow serving container does not provide Triton's performance optimizations.

Install MLflow on the SageMaker notebook instance only

Why wrong: This would track only exploratory work, not the training jobs.

Use the SageMaker Experiments integration with MLflow

Why wrong: SageMaker Experiments is a separate tracking service; integration with MLflow is not native.

Use SageMaker Processing to run MLflow after training

Why wrong: Post-processing is not real-time tracking; MLflow should be used during training.

SageMaker Multi-Model Endpoint

Why wrong: Multi-model endpoints do not natively support canary testing.

SageMaker Batch Transform

Why wrong: Batch Transform is for offline batch processing, not real-time inference.

SageMaker Serverless Inference

Why wrong: Serverless does not support canary deployments or multi-AZ configurations.

Use MLflow to track experiments and register models in MLflow's native registry, then export to SageMaker

Why wrong: The requirement is to register in SageMaker Model Registry, not MLflow's registry.

Use SageMaker Experiments for tracking, then manually register the model using SageMaker console

Why wrong: This does not leverage MLflow as required.

MLflow cannot be used with SageMaker; use SageMaker Experiments instead

Why wrong: MLflow can be used with SageMaker via the integration.

Create a custom container from scratch with Triton and deploy on SageMaker

Why wrong: Creating a custom container from scratch is not the specific SageMaker configuration required for Triton, as SageMaker provides native integration that handles the complex setup. This native support offers pre-built images and manages the intricate deployment of Triton, CUDA, and PyTorch, which is essential for models using custom CUDA kernels. Building from scratch would necessitate manual configuration of all these components, bypassing the intended SageMaker simplification. This option is tempting because custom containers offer maximum flexibility for deploying inference servers or models not natively supported by SageMaker, or when highly specific, non-standard customisation is needed.

Use a Multi-Model Endpoint with Triton

Why wrong: MME does not support Triton natively; Triton is typically deployed on a real-time endpoint.

Attach an Amazon Elastic Inference accelerator to the endpoint

Why wrong: Elastic Inference is not compatible with Triton; Triton manages its own GPU resources.

Set the StepStatus of successful steps to 'Cached'

Why wrong: This is not a direct action; 'Cached' status is automatically set when step caching is enabled and a cache hit occurs. The action to take is enabling caching.

Use parallel execution of pipeline steps

Why wrong: Parallel execution can improve speed but does not reuse outputs.

Create multiple pipeline versions for each run

Why wrong: Creating multiple pipeline versions does not reuse outputs from previous runs.

Disable caching for all steps to avoid unnecessary storage costs

Why wrong: Disabling caching would prevent reuse and increase costs.

SageMaker Neo and Amazon Elastic Inference cannot be used together

Why wrong: They can be used together; Neo compiles the model, and Elastic Inference provides the GPU.

SageMaker Neo provides a GPU acceleration service similar to Elastic Inference

Why wrong: Neo compiles models but does not provide acceleration hardware.

Amazon Elastic Inference compiles the model to run on GPU hardware

Why wrong: Elastic Inference does not compile; it provides hardware acceleration.

Real-time endpoint

Why wrong: Real-time endpoints are for low-latency, not for large offline datasets.

Asynchronous inference

Why wrong: Asynchronous is for near-real-time with large payloads, but not designed for batch processing of entire datasets.

Serverless inference

Why wrong: Serverless is for on-demand, low-latency inference, not batch processing.

S3 + SQS + SageMaker Batch Transform

Why wrong: Batch transform is for inference, not training.

S3 + Step Functions + SageMaker Training Job

Why wrong: Step Functions can orchestrate but the ML workflow is better managed by SageMaker Pipeline; Step Functions would require custom integration.

S3 + EventBridge + SageMaker Model Registry

Why wrong: Model Registry alone cannot execute the training pipeline; it requires a Pipeline or Step Functions.