MLA-C01 Deployment and Orchestration of ML Workflows Practice Test 7 — 15 Questions

Question 1

A company deploys a model using SageMaker and enables data capture for monitoring. After a week, they notice that the captured data is not being written to the specified S3 bucket. The endpoint is running and invocations are successful. What is the most likely cause?

Accepted Answer

The IAM role used for the endpoint does not have s3:PutObject permission for the capture bucket.. The most likely cause is that the IAM role associated with the SageMaker endpoint lacks the `s3:PutObject` permission for the target S3 bucket. Without this permission, the endpoint cannot write the captured inference data to S3, even though invocations succeed because the model itself does not require S3 write access to serve predictions.

Answer

The capture bucket is in a different region.

Answer

The endpoint is using a multi-model endpoint which does not support data capture.

Answer

The DataCaptureConfig parameter in the endpoint configuration is missing the "CaptureOptions" field.

Question 2

A team uses SageMaker Neo to compile a model for deployment on a target device. After compilation, they deploy the compiled model to a SageMaker endpoint using the Neo-optimized container. The endpoint fails to start with error "RuntimeError: Unable to load model". What could be the issue?

Accepted Answer

The target device architecture during compilation does not match the endpoint instance architecture.. Option D is correct because SageMaker Neo compiles a model for a specific target architecture (e.g., ARM, x86, GPU). When deploying the compiled model to a SageMaker endpoint, the endpoint instance type must have a CPU or accelerator architecture that matches the target device specified during compilation. If they do not match, the Neo-optimized runtime cannot load the compiled binary, resulting in a 'RuntimeError: Unable to load model'.

Answer

The compiled model was not uploaded to the correct S3 path.

Answer

The Neo compilation job failed silently.

Answer

The endpoint instance type does not support Neo.

Question 3

A company wants to use SageMaker to serve real-time predictions with a model that has a large memory footprint. They need to ensure the endpoint can handle traffic spikes. Which scaling policy should they use?

Accepted Answer

Target tracking policy. Target tracking scaling policy is the correct choice because it automatically adjusts the number of instances in the SageMaker endpoint based on a target metric, such as InvocationsPerInstance or ModelLatency, to handle traffic spikes without manual intervention. This policy is ideal for real-time inference with large memory models because it dynamically scales resources up or down to maintain the target metric, ensuring consistent performance during unpredictable traffic bursts.

Answer

Simple scaling policy

Answer

Scheduled scaling policy

Answer

Step scaling policy

Question 4

A data science team wants to deploy a real-time inference endpoint on Amazon SageMaker for a model that requires low latency (under 100 ms). The model is a small ensemble of three tree-based models, each about 50 MB. The team expects around 1000 requests per minute, with occasional spikes to 5000 requests per minute. Which instance type and deployment strategy would be MOST cost-effective while meeting the latency requirement?

Accepted Answer

Deploy a single model endpoint on an ml.c5.large instance with Auto Scaling configured using a target tracking policy based on invocations per minute. Option A is correct because deploying a single model endpoint on an ml.c5.large instance with Auto Scaling based on invocations per minute provides the necessary compute capacity for the expected 1000 requests per minute while scaling up to handle spikes up to 5000 requests per minute. The ml.c5.large instance offers sufficient memory (4 GB) and compute for three 50 MB tree-based models, and the target tracking policy ensures low latency by maintaining a buffer of capacity without over-provisioning, keeping inference under 100 ms.

Answer

Deploy a single model endpoint on an ml.c5.large instance with a Multi-Model endpoint

Answer

Use SageMaker batch transform with multiple ml.c5.large instances to process all requests offline

Answer

Deploy a single model endpoint on an ml.c5.xlarge instance with provisioned concurrency

Question 5

A company has a SageMaker endpoint that was deployed successfully and is in service. However, when the team sends test inferences using the InvokeEndpoint API, they receive a 500 internal server error. The endpoint logs in CloudWatch show a stack trace indicating 'OutOfMemoryError: Java heap space'. The model is a large XGBoost model (2 GB) and the endpoint is using an ml.m5.large instance with 8 GB of memory. What is the MOST likely cause and solution?

Accepted Answer

The instance type has insufficient memory for the model size; use a larger instance type like ml.m5.xlarge (16 GB) or ml.m5.2xlarge.. The OutOfMemoryError in Java heap space indicates that the model (2 GB) plus the runtime overhead of the XGBoost container and Java-based inference code exceed the available memory on the ml.m5.large instance (8 GB total, but not all is available for the Java heap). The most direct fix is to use a larger instance type, such as ml.m5.xlarge (16 GB) or ml.m5.2xlarge, to provide sufficient heap space for the model and inference operations.

Answer

The endpoint needs to have a smaller batch size configured in the real-time inference request.

Answer

The model is a Transformer model and requires a GPU instance; use ml.g4dn.xlarge instead.

Answer

The SageMaker container is not compatible with XGBoost; switch to a framework container.

Question 6

A company is running multiple SageMaker endpoints for different models, each serving a separate business unit. The total cost is growing rapidly. The ML engineering team wants to reduce costs without sacrificing performance or isolation. They are considering either consolidating models into a Multi-Model Endpoint (MME) or onto a Multi-Container Endpoint (MCE). The models vary in size from 100 MB to 5 GB, and traffic patterns are unpredictable. Which recommendation is MOST appropriate?

Accepted Answer

Use a Multi-Model Endpoint with a single large instance type to host all models, and enable SageMaker inference pipelines if pre-processing is needed.. A Multi-Model Endpoint (MME) is the most appropriate choice because it allows hosting multiple models on a single instance while keeping them isolated in separate memory spaces, which reduces cost by sharing the underlying infrastructure. MME dynamically loads and unloads models based on traffic, making it ideal for unpredictable patterns and model sizes ranging from 100 MB to 5 GB. Inference pipelines can be added for pre-processing without breaking the multi-model architecture, preserving performance and isolation.

Answer

Use Multi-Container Endpoints to deploy multiple models on a single instance.

Answer

Migrate all models to AWS Lambda functions for serverless inference.

Answer

Keep individual endpoints but switch to Graviton-based instances for cost savings.

Question 7

A company uses Amazon SageMaker to train and deploy machine learning models. They need to run batch predictions on 10 TB of data stored in Amazon S3 every night. The model is a PyTorch neural network that fits in GPU memory. The predictions are not time-sensitive, but the job must complete within 8 hours. Which approach would be the MOST cost-effective?

Accepted Answer

Use SageMaker batch transform with a single ml.p3.2xlarge instance.. Option D is the most cost-effective because SageMaker batch transform with a single ml.p3.2xlarge instance provides GPU acceleration for the PyTorch neural network, which fits in GPU memory, and can process 10 TB of data within 8 hours. Batch transform automatically handles data partitioning and inference, eliminating the need for custom orchestration, and the single instance avoids the overhead and cost of multiple instances. The ml.p3.2xlarge offers a balance of GPU compute and cost, making it ideal for non-time-sensitive nightly batch jobs.

Answer

Use SageMaker processing job with a script to load the model and run inference.

Answer

Create a real-time endpoint and send all data as a large batch.

Answer

Use multiple ml.c5.4xlarge instances in a batch transform job with custom partitioning.

Question 8

A machine learning engineer is configuring auto-scaling for a SageMaker real-time endpoint. The endpoint is expected to have steady traffic during business hours and low traffic at night. The engineer wants to minimize costs by scaling in during low traffic, but the model container has a long start-up time (about 5 minutes). Which scaling policy should the engineer use to prevent request drops during sudden traffic spikes?

Accepted Answer

Use a target tracking scaling policy based on average invocations per minute with a warm-up of 300 seconds.. Option B is correct because target tracking scaling policies in SageMaker automatically adjust capacity to maintain a target metric value, and the warm-up time of 300 seconds accounts for the 5-minute container start-up latency. This prevents request drops during sudden traffic spikes by ensuring new instances are fully initialized before they receive traffic, while still allowing the endpoint to scale in during low traffic to minimize costs.

Answer

Use a step scaling policy based on invocations per minute with a step that adds two instances at a time.

Answer

Use a scheduled scaling action to add instances before business hours and remove them after.

Answer

Use a simple scaling policy based on average CPU utilization with a cooldown period of 5 minutes.

Question 9

A company uses SageMaker endpoint with production variants for canary deployments. The team wants to gradually shift traffic from the old model variant (variant A) to the new model variant (variant B) over a period of 10 minutes. After the shift, if the new variant's error rate increases by more than 5%, they want to roll back automatically. Which solution meets these requirements with minimal manual intervention?

Accepted Answer

Use the SageMaker UpdateEndpoint API with a linear traffic shift from variant A to variant B over 10 minutes, and configure a CloudWatch alarm on the new variant's error rate that triggers a Lambda function to revert the traffic weights.. Option C is correct because the SageMaker UpdateEndpoint API supports a linear traffic shift between production variants, allowing you to gradually route traffic from variant A to variant B over a specified time period (here, 10 minutes). By attaching a CloudWatch alarm on the new variant's error rate that triggers a Lambda function to revert the traffic weights, you achieve automatic rollback with minimal manual intervention when the error rate exceeds the 5% threshold.

Answer

Use AWS Cloud Map to register the new variant and perform a slow rollout.

Answer

Deploy variant B as a separate endpoint and use Route 53 weighted routing to shift traffic.

Answer

Use AWS CodeDeploy with a deployment group to shift traffic and automatically roll back if CloudWatch alarms trigger.

Question 10

A data scientist needs to version and manage multiple models for a team of five. The team frequently experiments with different algorithms and hyperparameters. They need a centralized registry to store, deploy, and compare model versions. Which AWS service should the data scientist use?

Accepted Answer

Use Amazon SageMaker Model Registry.. Amazon SageMaker Model Registry is the correct choice because it provides a centralized repository specifically designed for cataloging, versioning, approving, and deploying machine learning models. It integrates natively with SageMaker pipelines and endpoints, enabling the team to compare model versions, manage metadata (e.g., hyperparameters, metrics), and promote models through stages (e.g., from staging to production) with approval workflows.

Answer

Store each model artifact in Amazon S3 with manual versioning in the key name.

Answer

Use AWS Config to track model version changes.

Answer

Use AWS CodeArtifact to store model packages.

Question 11

An ML team uses AWS Step Functions to orchestrate a multi-step inference pipeline: data preprocessing, model inference, and postprocessing. The pipeline runs on demand for single records. The team notices that the pipeline occasionally fails due to timeouts in the preprocessing step. They want to implement retries with exponential backoff and a maximum retry count of 3 for that step. How should they configure this?

Accepted Answer

Modify the Step Functions state machine definition to add a Retry field on the preprocessing state with a maximum retry count of 3 and an exponential backoff rate of 2.0.. Option B is correct because AWS Step Functions natively supports retry logic with exponential backoff directly in the state machine definition. By adding a `Retry` field on the preprocessing state with `MaxAttempts: 3` and `BackoffRate: 2.0`, the service automatically retries the step on specified errors (e.g., `States.Timeout` or `Lambda.ServiceException`) with exponentially increasing wait times, without requiring custom code or external orchestration.

Answer

Implement retry logic inside the preprocessing Lambda function code.

Answer

Wrap the preprocessing step in a SageMaker Pipeline step with retry policy.

Answer

Add a Catch in the state machine to rerun the entire pipeline if preprocessing fails.

Question 12

A company is deploying a deep learning model for real-time inference using Amazon SageMaker. The model is a CPU-intensive XGBoost model that performs well with CPU. However, the team wants to minimize latency further by using hardware acceleration. They are considering Amazon Elastic Inference (EI) or moving to a GPU instance. The model is not optimized for GPU, so significant code changes would be required. Which approach is the MOST cost-effective way to reduce latency without changing the model code?

Accepted Answer

Attach an Elastic Inference accelerator (e.g., ml.eia2.medium) to the existing CPU endpoint.. Option B is correct because Amazon Elastic Inference (EI) allows you to attach a low-cost GPU-powered acceleration to an existing SageMaker CPU endpoint without any code changes. Since the XGBoost model is CPU-optimized and not GPU-native, EI provides hardware acceleration for the inference computation (specifically matrix operations) while keeping the model execution on the CPU, thus reducing latency without requiring model modifications.

Answer

Use a GPU instance (ml.p3.2xlarge) and optimize the model with SageMaker Neo compilation.

Answer

Use SageMaker Neo to compile the model for CPU with INT8 quantization.

Answer

Migrate the model to AWS Lambda with a custom runtime and use AVX instructions.

Question 13

An ML team is running multiple SageMaker endpoints for various models. The monthly cost is higher than expected. Which TWO actions would help reduce costs without negatively impacting performance?

Accepted Answer

Consolidate multiple small models into a single Multi-Model Endpoint on a larger instance.. Option A is correct because SageMaker Multi-Model Endpoints allow you to host multiple small models on a single endpoint behind a common serving container, sharing the underlying instance resources. This reduces the number of endpoints and instances needed, lowering costs without degrading performance, as models are loaded and unloaded dynamically based on traffic.

Answer

Increase the number of minimum instances to handle traffic spikes without scaling.

Answer

Limit the maximum number of concurrent invocations per endpoint.

Answer

Use a scheduled scaling to turn off endpoints during non-business hours.

Question 14

A company deploys a SageMaker endpoint that is InService, but inference requests are returning 503 Service Unavailable errors when traffic is high. The endpoint uses three ml.m5.large instances with target tracking scaling based on CPU utilization. The team has confirmed the model container is healthy. Which TWO possible issues could cause 503 errors?

Accepted Answer

The instance memory is insufficient for the model, causing the container to run out of memory under load.. Option B is correct because the endpoint might not have enough instances to handle peak load if scaling policies have cooldown delays. Option E is correct because if the model is large and instances run out of memory, new requests may be rejected. Option A is wrong because execution role permissions cause 403 or 500 errors, not 503. Option C is wrong because 503 indicates server overload, not client request timeout. Option D is wrong because 400 errors are from client side.

Answer

The model output is too large for the response buffer.

Answer

The execution role does not have permission to invoke SageMaker endpoint.

Answer

The inference request size exceeds the maximum payload size limit.

Question 15

A company is adopting Amazon SageMaker Pipelines to automate their ML workflow. They want to choose three key benefits that SageMaker Pipelines provides over traditional manual scripts and ad-hoc steps. Which THREE benefits are correct?

Accepted Answer

Model lineage tracking from raw data to trained model artifacts.. Option A is correct because SageMaker Pipelines automatically captures and tracks the lineage of every artifact, including datasets, processing jobs, training jobs, and model versions. This lineage is stored in SageMaker's metadata store, enabling full traceability from raw data to the final model artifact, which is critical for auditability and compliance in ML workflows.

Answer

Automated deployment of models to endpoints upon pipeline completion.

Answer

Automatic scaling of compute resources based on data volume.