MLS-C01 Machine Learning Implementation and Operations Practice Test 11 — 15 Questions

Question 1

A company's ML pipeline uses AWS Step Functions to orchestrate data preprocessing, training, and evaluation. The training step occasionally fails due to a transient error. What is the most robust way to handle this without manual intervention?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Implement a retry policy with exponential backoff on the training step in the state machine. AWS Step Functions natively supports retry policies with exponential backoff, which automatically retries failed tasks after a delay that increases progressively. This handles transient errors (e.g., resource contention, network glitches) without manual intervention, making the pipeline robust and self-healing.

Answer

Configure a CloudWatch alarm to notify the team when the step fails

Answer

Use a parallel state to run multiple training instances simultaneously

Answer

Use a custom Lambda function to catch the error and restart the training step

Question 2

A data scientist needs to perform hyperparameter optimization for a gradient boosting model. Which built-in Amazon SageMaker feature should they use?

Accepted Answer

Amazon SageMaker Automatic Model Tuning. Amazon SageMaker Automatic Model Tuning (A) is the built-in feature specifically designed for hyperparameter optimization. It automates the search for the best combination of hyperparameters by launching multiple training jobs with different hyperparameter values, using strategies like Bayesian optimization, random search, or Hyperband. This directly addresses the data scientist's need to optimize a gradient boosting model's hyperparameters.

Answer

Amazon SageMaker Clarify

Answer

Amazon SageMaker Debugger

Answer

Amazon SageMaker Neo

Question 3

A company's ML model is deployed on a SageMaker endpoint. The model's predictions are used in a customer-facing application that requires low latency. Over time, the model's performance degrades due to data drift. What is the most suitable approach to detect this drift automatically?

Accepted Answer

Enable Amazon SageMaker Model Monitor to continuously check for data drift. Amazon SageMaker Model Monitor is purpose-built to automatically detect data drift by continuously comparing incoming inference data against a baseline dataset. It computes statistical metrics (e.g., distribution distances like Kolmogorov-Smirnov or Chi-squared) and raises alerts when drift exceeds configurable thresholds, enabling proactive retraining without manual intervention. This directly addresses the need for automated drift detection in a low-latency customer-facing application.

Answer

Set up a CloudWatch alarm on the endpoint's invocation latency

Answer

Periodically retrain the model using all historical data

Answer

Use Amazon S3 events to trigger a Lambda function that compares distributions

Question 4

An ML team is using SageMaker Processing jobs to run feature engineering scripts. The scripts require a specific Python package not included in the default SageMaker image. How should the team provide this package?

Accepted Answer

Create a custom Docker image that includes the package and use it for the Processing job. SageMaker Processing jobs run in isolated Docker containers, and the default SageMaker images only include pre-installed packages. To add a custom Python package, the team must create a custom Docker image that includes the package (e.g., via a Dockerfile with 'pip install <package>'), then specify that image URI in the Processing job configuration. This ensures the package is available in the container environment before the script executes.

Answer

Include 'pip install <package>' in the processing script

Answer

Use the SageMaker prebuilt deep learning container with the package

Answer

Place a requirements.txt file in the input data S3 bucket

Question 5

Which TWO options are valid ways to reduce inference latency for a model deployed on a SageMaker real-time endpoint? (Select TWO.)

Accepted Answer

Enable SageMaker Neo to compile the model for the target instance. SageMaker Neo compiles the trained model into an optimized binary for the specific target instance type, using hardware-specific instructions (e.g., Intel MKL-DNN, NVIDIA TensorRT) to reduce inference latency without sacrificing accuracy. This compilation optimizes the model graph and fuses operations, leading to faster execution on the deployed endpoint.

Answer

Use SageMaker batch transform instead of real-time endpoint

Answer

Deploy the model to multiple instances behind a load balancer

Answer

Increase the endpoint's invocation timeout

Question 6

Which THREE steps should be taken to secure a SageMaker notebook instance that accesses sensitive data? (Select THREE.)

Accepted Answer

Enable encryption at rest for the notebook's EBS volume. SageMaker notebook instances use an Amazon EBS volume for storage, and enabling encryption at rest for this volume ensures that sensitive data stored on the notebook (e.g., datasets, model artifacts) is encrypted using AWS KMS-managed keys. This protects data at the storage layer, which is a fundamental security requirement for compliance with standards like HIPAA or PCI DSS.

Answer

Grant root access to the notebook instance for flexibility

Answer

Allow direct internet access from the notebook for downloading packages

Question 7

Which TWO AWS services can be used to deploy a trained model for serverless inference? (Select TWO.)

Accepted Answer

AWS Lambda with a container image. AWS Lambda with a container image allows you to package a trained model and its dependencies into a Docker container and deploy it as a serverless function. Lambda automatically scales the inference endpoint in response to incoming requests, and you pay only for the compute time consumed during inference, with no idle infrastructure costs.

Answer

Amazon SageMaker batch transform

Answer

Amazon Elastic Container Service (ECS) with Fargate

Answer

Amazon EC2 instances

Question 8

A company runs a real-time fraud detection model on a SageMaker endpoint. The model is a TensorFlow neural network trained on transactional data. The endpoint uses a single ml.p3.2xlarge instance. Recently, the application’s latency has increased from 50ms to 500ms on average. The CloudWatch metrics show that CPU utilization is at 90%, GPU utilization is at 30%, and memory utilization is at 40%. The number of requests per second has remained stable. The ML team suspects the model is not fully utilizing the GPU. What action should the team take to reduce latency without changing the instance type?

Accepted Answer

Use SageMaker Neo to compile the model for the target instance. SageMaker Neo compiles the model to optimize inference for the target hardware, improving GPU utilization and reducing latency. Option A is incorrect because SageMaker Batch Transform is for offline inference, not real-time requests. Option B is incorrect because switching to a CPU-based instance (ml.c5.large) would not leverage the GPU and could increase latency. Option D is incorrect because adding more instances improves throughput, not per-request latency, and does not address GPU underutilization.

Answer

Switch to SageMaker Batch Transform to process requests in batches

Answer

Change the endpoint to a compute-optimized instance like ml.c5.large

Answer

Increase the number of instances behind the endpoint and use a load balancer

Question 9

A company is deploying a machine learning model to production on Amazon SageMaker. The model requires low-latency inference (under 10 ms) for real-time predictions. The data scientist has trained a model using XGBoost and wants to minimize cost while meeting latency requirements. Which SageMaker hosting option should be used?

Accepted Answer

Use a real-time endpoint with a single model. A real-time endpoint with a single model is the correct choice because it provides dedicated, always-on compute resources that can consistently achieve sub-10 ms inference latency for XGBoost models. SageMaker real-time endpoints keep instances warm and route requests directly to the model container, minimizing cold-start delays and network overhead, which is essential for low-latency requirements.

Answer

Use a serverless inference endpoint

Answer

Use a real-time endpoint with multi-model hosting

Answer

Use a batch transform job

Answer

Use an asynchronous inference endpoint

Question 10

A data scientist is training a deep learning model on Amazon SageMaker using a custom TensorFlow container. The training job fails with an OutOfMemory error. The instance type is ml.p3.2xlarge with 16 GB GPU memory and 61 GB system memory. The model uses mixed precision training. Which step should the data scientist take to resolve the issue without changing the instance type?

Accepted Answer

Reduce the batch size. Reducing the batch size directly decreases the memory footprint per training step, which is the most straightforward way to resolve an OutOfMemory error without changing the instance type. Since the model already uses mixed precision training (which reduces memory usage via FP16), the remaining memory pressure is likely from the batch size being too large for the 16 GB GPU memory on the ml.p3.2xlarge instance.

Answer

Use gradient accumulation to simulate a larger batch size

Answer

Use model parallelism across multiple GPUs

Answer

Enable automatic mixed precision (AMP)

Answer

Increase the instance type to ml.p3.8xlarge

Question 11

A company is using Amazon SageMaker to train a model. The training data is stored in an S3 bucket encrypted with AWS KMS. The SageMaker training role has the necessary permissions to decrypt the data. However, the training job fails with an access denied error. What is the most likely cause?

Accepted Answer

The training role does not have kms:Decrypt permission for the KMS key. The error message 'access denied' during a SageMaker training job with KMS-encrypted S3 data typically indicates that the training role lacks the kms:Decrypt permission for the specific KMS key used to encrypt the S3 objects. Even if the role has S3 read permissions (s3:GetObject), SageMaker must decrypt the data before reading it, which requires explicit KMS key policy or IAM policy granting kms:Decrypt. Without this, the training job fails with an access denied error.

Answer

The S3 bucket policy does not grant access to the training role

Answer

The training image is not compatible with encrypted data

Answer

CloudTrail logging is disabled

Answer

The training job is not in the same VPC as the S3 bucket

Question 12

A machine learning team is deploying a model using Amazon SageMaker. They need to automatically retrain the model every week with new data and update the endpoint without downtime. Which approach should they use?

Accepted Answer

Use SageMaker Pipelines to automate retraining and deploy a new endpoint with blue/green deployment. The correct approach is to use SageMaker Pipelines, which provides a fully managed service to automate the machine learning workflow, including retraining on a schedule. SageMaker Pipelines supports blue/green deployment patterns using `CreateEndpointConfig` and `UpdateEndpoint` to update the endpoint without downtime. Option E correctly describes this. Option D is incorrect because automatic model tuning is for hyperparameter optimization, not scheduled retraining. Option C (AWS Lambda) could be used, but it requires more manual orchestration and is not the best practice recommended by AWS.

Answer

Use SageMaker Ground Truth to label new data and trigger retraining

Answer

Use SageMaker batch transform to periodically generate predictions and replace the model

Answer

Use AWS Lambda to trigger retraining on a schedule and deploy a new endpoint

Answer

Use SageMaker automatic model tuning with a schedule and update the endpoint using CreateEndpointConfig and UpdateEndpoint

Question 13

A company is running a real-time inference endpoint on Amazon SageMaker. The endpoint is using an ml.c5.xlarge instance. Over the past month, the CPU utilization has been consistently below 10%, and the latency is well within requirements. The company wants to reduce costs. What should they do?

Accepted Answer

Use a smaller instance type. The CPU utilization is consistently below 10%, indicating significant over-provisioning. Downgrading to a smaller instance type (e.g., ml.c5.large or ml.t3.medium) directly reduces the per-hour cost while still meeting the latency requirements. This is the most straightforward cost optimization when the current instance is underutilized and performance is already satisfactory.

Answer

Set up a scaling policy to scale down to zero

Answer

Switch to a multi-model endpoint

Answer

Use a batch transform job instead

Answer

Move to a serverless inference endpoint

Question 14

A data scientist is using Amazon SageMaker to train a model. The training job is taking longer than expected. The data scientist notices that the GPU utilization is low. Which action would most likely improve GPU utilization?

Accepted Answer

Increase the batch size. Low GPU utilization during training often indicates that the GPU is waiting for data to process, a condition known as data bottleneck. Increasing the batch size allows the GPU to process more samples per forward/backward pass, keeping it busier and improving utilization. In SageMaker, this directly impacts the training loop by reducing the frequency of data loading and model update steps.

Answer

Change to a CPU-based instance

Answer

Decrease the batch size

Answer

Use a larger instance type

Answer

Enable data augmentation

Question 15

A company is using Amazon SageMaker to deploy a model for real-time predictions. The model requires access to a DynamoDB table to look up features. The SageMaker endpoint is configured with a VPC and subnet. However, the endpoint cannot connect to DynamoDB. What is the most likely reason?

Accepted Answer

The VPC does not have a VPC endpoint for DynamoDB or a NAT gateway. A SageMaker endpoint deployed in a VPC, by default, cannot access public AWS services like DynamoDB unless the VPC has a VPC endpoint for DynamoDB (Gateway endpoint) or a NAT gateway to route traffic through an internet gateway. Without either, the endpoint's private subnet has no route to DynamoDB's public endpoints, causing the connection failure.

Answer

The security group does not allow outbound traffic to DynamoDB

Answer

The IAM role for the endpoint does not have dynamodb:GetItem permission

Answer

The DynamoDB table is in a different AWS Region

Answer

The CloudWatch logs show no errors