MLS-C01 Machine Learning Implementation and Operations Practice Test 6 — 15 Questions

Question 1

A company is deploying a machine learning model to production on Amazon SageMaker. The model requires low-latency inference (under 10 ms) for real-time predictions. The data scientist has trained a model using XGBoost and wants to minimize cost while meeting latency requirements. Which SageMaker hosting option should be used?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use a real-time endpoint with a single model. A real-time endpoint with a single model is the correct choice because it provides dedicated, always-on compute resources that can consistently achieve sub-10 ms inference latency for XGBoost models. SageMaker real-time endpoints keep instances warm and route requests directly to the model container, minimizing cold-start delays and network overhead, which is essential for low-latency requirements.

Answer

Use a serverless inference endpoint

Answer

Use a real-time endpoint with multi-model hosting

Answer

Use a batch transform job

Answer

Use an asynchronous inference endpoint

Question 2

A data scientist is training a deep learning model on Amazon SageMaker using a custom TensorFlow container. The training job fails with an OutOfMemory error. The instance type is ml.p3.2xlarge with 16 GB GPU memory and 61 GB system memory. The model uses mixed precision training. Which step should the data scientist take to resolve the issue without changing the instance type?

Accepted Answer

Reduce the batch size. Reducing the batch size directly decreases the memory footprint per training step, which is the most straightforward way to resolve an OutOfMemory error without changing the instance type. Since the model already uses mixed precision training (which reduces memory usage via FP16), the remaining memory pressure is likely from the batch size being too large for the 16 GB GPU memory on the ml.p3.2xlarge instance.

Answer

Use gradient accumulation to simulate a larger batch size

Answer

Use model parallelism across multiple GPUs

Answer

Enable automatic mixed precision (AMP)

Answer

Increase the instance type to ml.p3.8xlarge

Question 3

A company is using Amazon SageMaker to train a model. The training data is stored in an S3 bucket encrypted with AWS KMS. The SageMaker training role has the necessary permissions to decrypt the data. However, the training job fails with an access denied error. What is the most likely cause?

Accepted Answer

The training role does not have kms:Decrypt permission for the KMS key. The error message 'access denied' during a SageMaker training job with KMS-encrypted S3 data typically indicates that the training role lacks the kms:Decrypt permission for the specific KMS key used to encrypt the S3 objects. Even if the role has S3 read permissions (s3:GetObject), SageMaker must decrypt the data before reading it, which requires explicit KMS key policy or IAM policy granting kms:Decrypt. Without this, the training job fails with an access denied error.

Answer

The S3 bucket policy does not grant access to the training role

Answer

The training image is not compatible with encrypted data

Answer

CloudTrail logging is disabled

Answer

The training job is not in the same VPC as the S3 bucket

Question 4

A machine learning team is deploying a model using Amazon SageMaker. They need to automatically retrain the model every week with new data and update the endpoint without downtime. Which approach should they use?

Accepted Answer

Use SageMaker Pipelines to automate retraining and deploy a new endpoint with blue/green deployment. The correct approach is to use SageMaker Pipelines, which provides a fully managed service to automate the machine learning workflow, including retraining on a schedule. SageMaker Pipelines supports blue/green deployment patterns using `CreateEndpointConfig` and `UpdateEndpoint` to update the endpoint without downtime. Option E correctly describes this. Option D is incorrect because automatic model tuning is for hyperparameter optimization, not scheduled retraining. Option C (AWS Lambda) could be used, but it requires more manual orchestration and is not the best practice recommended by AWS.

Answer

Use SageMaker Ground Truth to label new data and trigger retraining

Answer

Use SageMaker batch transform to periodically generate predictions and replace the model

Answer

Use AWS Lambda to trigger retraining on a schedule and deploy a new endpoint

Answer

Use SageMaker automatic model tuning with a schedule and update the endpoint using CreateEndpointConfig and UpdateEndpoint

Question 5

A company is using Amazon SageMaker to deploy a model for real-time predictions. The model requires access to a DynamoDB table to look up features. The SageMaker endpoint is configured with a VPC and subnet. However, the endpoint cannot connect to DynamoDB. What is the most likely reason?

Accepted Answer

The VPC does not have a VPC endpoint for DynamoDB or a NAT gateway. A SageMaker endpoint deployed in a VPC, by default, cannot access public AWS services like DynamoDB unless the VPC has a VPC endpoint for DynamoDB (Gateway endpoint) or a NAT gateway to route traffic through an internet gateway. Without either, the endpoint's private subnet has no route to DynamoDB's public endpoints, causing the connection failure.

Answer

The security group does not allow outbound traffic to DynamoDB

Answer

The IAM role for the endpoint does not have dynamodb:GetItem permission

Answer

The DynamoDB table is in a different AWS Region

Answer

The CloudWatch logs show no errors

Question 6

A data scientist is training a model using Amazon SageMaker. The training dataset is 500 GB and is stored in S3. The data scientist wants to use Pipe input mode to stream data directly from S3 to the training container. However, the training job fails with an error indicating that the container cannot read the data. What is the most likely cause?

Accepted Answer

The training algorithm does not support Pipe mode. Pipe mode in SageMaker streams data directly from S3 to the training container via a FIFO pipe, but the training algorithm must be specifically designed to read from a pipe (e.g., via stdin or a named pipe). If the algorithm expects random access to files or uses libraries that require seekable file handles (like many image-processing or custom Python scripts using `open()`), it will fail because a pipe is a sequential, non-seekable stream. SageMaker’s built-in algorithms like BlazingText and XGBoost support Pipe mode, but custom algorithms often do not unless explicitly coded for it.

Answer

The training instance does not have enough memory

Answer

The IAM role does not have s3:GetObject permission

Answer

The data is compressed and Pipe mode cannot handle compressed data

Answer

The S3 bucket is in a different Region

Question 7

Refer to the exhibit. A company is using an IAM role with the attached policy to deploy a SageMaker model. The data scientist can create training jobs and models, but when trying to create an endpoint, they receive an access denied error. What is the missing permission?

Accepted Answer

ec2:CreateNetworkInterface. When SageMaker creates an endpoint, it provisions a network interface in the customer's VPC to route traffic to the endpoint instances. The IAM role must have the ec2:CreateNetworkInterface permission to allow SageMaker to create this ENI; without it, the endpoint creation fails with an access denied error.

Answer

cloudwatch:PutMetricData

Answer

iam:PassRole

Answer

sagemaker:InvokeEndpoint

Answer

kms:Decrypt

Question 8

Refer to the exhibit. A data scientist is reviewing CloudWatch logs for a SageMaker real-time endpoint. The log shows that a prediction took 15 ms. The endpoint is configured with an ml.c5.large instance and the model is a small scikit-learn model. The latency requirement is under 10 ms. Which action would most likely reduce the latency?

Accepted Answer

Use a larger instance type. The latency of 15 ms exceeds the 10 ms requirement, indicating that the current ml.c5.large instance lacks sufficient compute resources (CPU) to process predictions quickly enough. Upgrading to a larger instance type (e.g., ml.c5.xlarge or ml.c5.2xlarge) provides more CPU capacity, reducing inference time by allowing the model to compute predictions faster. This directly addresses the bottleneck for a small scikit-learn model, which is CPU-bound and benefits from increased compute power.

Answer

Add more instances to the endpoint

Answer

Change the model to a TensorFlow model

Answer

Enable SageMaker Batch Transform

Answer

Increase the batch size for inference

Question 9

A company uses Amazon SageMaker to train models. The data scientist wants to automate the retraining process whenever new data arrives in an S3 bucket. Which THREE services can be used together to achieve this? (Choose THREE.)

Accepted Answer

Amazon S3. Amazon S3 (A) is correct because it is the source of truth for the training data and can emit s3:ObjectCreated:* event notifications when new objects arrive, which is the trigger for the automation. AWS Lambda (C) is correct because it can be configured as the target of the S3 event notification and run code that starts the retraining job, for example by calling the SageMaker CreateTrainingJob API. Amazon SageMaker (D) is correct because it is the service that actually performs the model retraining, either via CreateTrainingJob or by starting a SageMaker Pipelines execution. Amazon EC2 (B) is not needed since Lambda provides the serverless compute for the trigger logic, and AWS Glue (E) is an ETL service that is not required to move or transform the data in this scenario.

Answer

Amazon EC2

Answer

AWS Glue

Question 10

A company operates a real-time fraud detection system using an Amazon SageMaker endpoint. The model is a gradient boosting model trained on historical transaction data. The endpoint is deployed on an ml.c5.2xlarge instance with auto-scaling enabled based on average latency. Recently, during a flash sale event, the endpoint started returning HTTP 503 errors. The CloudWatch metrics show that the CPU utilization is at 70%, and the average latency has increased from 50 ms to 200 ms. The auto-scaling policy is configured to add one instance when average latency exceeds 100 ms for 5 consecutive minutes, and remove one instance when latency drops below 50 ms for 5 minutes. The current number of instances is 2. The flash sale lasted 30 minutes. What should the company do to prevent this issue in future flash sales?

Accepted Answer

Pre-warm the endpoint by setting a minimum number of instances that can handle the expected peak load before the flash sale. The endpoint returned 503s because auto-scaling based on latency is reactive and too slow: the policy needs 5 consecutive minutes above 100 ms before adding a single instance, and the flash sale lasted only 30 minutes. Pre-warming the endpoint by setting a minimum instance count sized for the expected peak load ensures capacity is already in place before traffic spikes, eliminating the cold-start scaling lag. This is the standard SageMaker pattern for predictable traffic events.

Answer

Enable request throttling to drop excess requests

Answer

Change the instance type to ml.c5.4xlarge to handle higher load

Answer

Change the model to a simpler model with lower latency

Question 11

A company has deployed a machine learning model on a SageMaker endpoint that serves predictions to a web application. The model uses a custom inference container that loads the model artifacts from an ECR repository. After updating the model with new training data, the data scientist creates a new model and updates the endpoint. However, some users report that they still get predictions from the old model. The data scientist confirms that the endpoint configuration points to the new model. What is the most likely cause?

Accepted Answer

DNS caching on the client side is resolving to the old endpoint IP address. The most likely cause is DNS caching on the client side. When the endpoint is updated, SageMaker may change the underlying endpoint's DNS name or IP address. If clients have cached the old DNS resolution, they will continue to send requests to the old endpoint, which may still be serving the old model if the old endpoint is not immediately terminated. SageMaker endpoints have a DNS name that resolves to the endpoint's IP addresses. During an update, the endpoint configuration is changed, but the DNS name remains the same. However, if the update involves replacing instances, the IP addresses may change, and DNS caching could cause clients to use stale IPs. Alternatively, if the endpoint is updated in place, the old model may still be served until the update completes. But the question states that the endpoint configuration points to the new model, so the issue is likely client-side DNS caching.

Answer

The new model artifacts are not correctly uploaded to S3

Answer

The endpoint is behind a load balancer that is not updated

Answer

The inference container is cached and not pulling the new image

Question 12

A company is deploying a real-time inference endpoint using SageMaker. The model has a high memory footprint and requires GPU acceleration. Which instance type and configuration should be used to minimize cost while meeting latency requirements?

Accepted Answer

ml.p3.2xlarge with 1 GPU. (ml.p3.2xlarge with 1 GPU) is correct because it provides the required GPU acceleration for the high-memory-footprint model while using the smallest instance in the P3 family, which minimizes cost. The P3 instances use NVIDIA V100 GPUs with high memory bandwidth, suitable for real-time inference with low latency, and the 2xlarge size offers sufficient GPU memory without over-provisioning.

Answer

ml.g4dn.xlarge with 1 GPU

Answer

ml.c5.xlarge with no GPU

Answer

ml.p3.16xlarge with 8 GPUs

Question 13

A company is using SageMaker Ground Truth to label images for a computer vision model. After launching the labeling job, they notice that the labeling throughput is lower than expected. What should they do to increase throughput?

Accepted Answer

Use a private workforce with more workers.. SageMaker Ground Truth labeling throughput is primarily limited by the number of workers available to process tasks. Using a private workforce allows you to directly control and scale the number of workers, which increases parallelism and overall throughput. Public or vendor workforces have fixed capacity and may not scale as quickly, so adding more private workers is the most effective way to boost throughput.

Answer

Change the labeling task to use a single annotator per image.

Answer

Reduce the number of workers assigned to each task.

Answer

Increase the time allowed for each labeling task.

Question 14

A company is training a deep learning model on SageMaker using multiple GPUs. The training is slow due to inefficient data loading. Which TWO actions can improve I/O performance?

Accepted Answer

Use Pipe mode input for training data.. Pipe mode streams training data directly from Amazon S3 to the training algorithm, bypassing the need to download data to disk before training begins. This eliminates disk I/O bottlenecks and reduces data loading latency, which is critical for GPU-intensive training where GPUs may otherwise idle waiting for data.

Answer

Use instance store volumes for data.

Answer

Increase the instance count to a single large instance.

Answer

Use Amazon EBS volumes attached to training instances.

Question 15

A company is using SageMaker Autopilot to automatically build ML models. They want to ensure that the generated models are reproducible. Which TWO settings should they configure?

Accepted Answer

Set a random seed.. Option A is correct because setting a random seed fixes the pseudo-random number generation used during data splitting, model initialization, and hyperparameter search, which is essential for obtaining identical results across repeated Autopilot runs. Option B is correct because specifying a validation split (e.g., via ValidationFraction or a defined validation dataset) ensures the same data is consistently held out for model evaluation, removing variability caused by different train/validation partitions. Option C is not required for reproducibility because running multiple trials (candidate models) increases exploration but does not by itself guarantee identical outcomes. Option D is not required because early stopping affects training duration and can introduce run-to-run variability rather than eliminate it. Option E is not required because automatic feature engineering transforms data but does not ensure deterministic, repeatable model generation on its own.

Answer

Use multiple trials.

Answer

Enable early stopping.

Answer

Enable automatic feature engineering.