MLS-C01 Machine Learning Implementation and Operations Practice Test 10 — 15 Questions

Question 1

A team is training an XGBoost model using SageMaker with a large dataset in S3 (100 GB). Training is taking too long. Which change will most likely reduce training time without sacrificing accuracy?

Accepted Answer

Configure Pipe mode for data input. Pipe mode streams training data directly from S3 into the SageMaker training container instead of downloading the full dataset to local EBS first, which dramatically reduces startup time for large datasets like 100 GB. It preserves accuracy because the same data is used — only the ingestion method changes.

Answer

Reduce the number of training instances

Answer

Enable SageMaker Managed Spot Training

Answer

Use a larger instance type with more vCPUs

Question 2

A company deploys a SageMaker model for inference. After a few days, response times increase significantly. CloudWatch metrics show high CPU utilization and memory usage. The model is a large ensemble. What is the most cost-effective solution?

Accepted Answer

Configure SageMaker automatic scaling based on CPU utilization. SageMaker automatic scaling based on CPU utilization is the most cost-effective solution because it dynamically adjusts the number of inference instances in response to real-time demand, adding capacity only when CPU usage is high and removing it when demand drops. This avoids over-provisioning while maintaining performance for the large ensemble model, which is compute-intensive. Other options either introduce manual overhead, are unsuitable for large models, or incur unnecessary cost by permanently using larger instances.

Answer

Use CloudWatch alarms to notify the team, who manually launch additional endpoints

Answer

Migrate the model to AWS Lambda with provisioned concurrency

Answer

Replace the current instance type with a larger one

Question 3

A team uses SageMaker to train a deep learning model. They notice the training job is using only a fraction of the GPU memory. Which configuration change would most improve GPU utilization?

Accepted Answer

Increase the batch size in the training script. Increasing the batch size allows each training step to process more data samples simultaneously, which increases the computational load per step and better saturates the GPU's parallel processing units. This directly improves GPU memory utilization because larger batches keep more tensors resident in memory and increase the arithmetic intensity of matrix operations, reducing idle time.

Answer

Decrease the batch size to reduce memory fragmentation

Answer

Use a single GPU instead of multiple GPUs

Answer

Enable SageMaker Managed Spot Training

Question 4

A company wants to serve a scikit-learn model via SageMaker. The inference code requires a custom preprocessing step that is not in the default scikit-learn container. What is the simplest way to deploy?

Accepted Answer

Create a custom Docker image extending the SageMaker scikit-learn container. Extending the SageMaker scikit-learn container with a custom Docker image is the simplest and most direct way to add custom preprocessing logic that is not included in the default container. SageMaker's pre-built scikit-learn container supports only standard scikit-learn inference code; any additional dependencies or custom preprocessing steps require you to build a custom image that inherits from the official SageMaker scikit-learn image and adds your code. This approach avoids the complexity of managing separate inference pipelines or external services.

Answer

Package the code in a Lambda layer and use SageMaker hosting

Answer

Use SageMaker Batch Transform with a custom processing script

Answer

Use SageMaker Neo to compile the model and add preprocessing

Question 5

An ML team is deploying a model for real-time inference. They require A/B testing to compare a new model against the existing one. Which THREE steps should they take to set up this test?

Accepted Answer

Set up a second production variant for the existing model. To perform A/B testing with SageMaker, you must create a second production variant for the existing model so that both the old and new models are deployed as separate variants under the same endpoint. This allows the endpoint to serve both models simultaneously and compare their performance.

Answer

Set up a SageMaker Batch Transform job for each model

Answer

Configure CloudWatch alarms to trigger variant switching

Question 6

A company is using SageMaker to train a model and wants to ensure that the training data is encrypted at rest and in transit, and that the trained model artifacts are also encrypted. Which THREE actions should the company take?

Accepted Answer

Specify a KMS key in the SageMaker training job configuration to encrypt the ML storage volume. Option A is correct because specifying a KMS key in the SageMaker training job configuration encrypts the ML storage volume attached to the training instances, protecting data at rest on the instance storage. Option B is correct because enabling SageMaker model encryption with a KMS key ensures the trained model artifacts stored in S3 are encrypted at rest. Option E is correct because enabling S3 server-side encryption with SSE-KMS on the training data bucket encrypts the input training data at rest. Option C is not correct because running the training job in a VPC without internet access addresses network isolation, not encryption at rest or in transit. Option D is not correct because CloudTrail logs API activity for auditing, not encryption of data or model artifacts.

Answer

Configure the training job to run in a VPC with no internet access

Answer

Enable AWS CloudTrail to log all API calls

Question 7

A data scientist has this IAM policy attached to their IAM role. They are trying to run a SageMaker training job that reads data from 'my-bucket' and writes output to 'my-bucket'. The job fails. What is the most likely reason?

Accepted Answer

Missing iam:PassRole permission. The most likely reason the SageMaker training job fails is that the IAM role lacks the `iam:PassRole` permission. When SageMaker creates a training job, it must assume the execution role specified in the request; without `iam:PassRole`, the service cannot pass the role to itself, causing the API call to fail. This is a common prerequisite for any SageMaker job that requires an execution role.

Answer

The sagemaker:CreateTrainingJob action is not allowed on specific resources

Answer

Missing s3:ListBucket permission on the bucket

Answer

The training job requires permissions to write to CloudWatch Logs

Question 8

A company deploys a machine learning model on Amazon SageMaker for real-time inference. The model receives requests with large payloads (up to 5 MB) and the inference latency is high. Which configuration change would MOST likely reduce latency?

Accepted Answer

Use a larger instance type with more memory and compute. Increasing the instance type to one with more memory and compute directly addresses the bottleneck caused by large payloads (up to 5 MB) and high inference latency. SageMaker real-time endpoints process requests synchronously, so a larger instance provides more CPU/GPU and memory bandwidth to serialize/deserialize and process the payload faster, reducing overall latency.

Answer

Pre-load multiple model containers on the same endpoint

Answer

Reduce the batch size for inference requests

Answer

Enable payload compression using SageMaker built-in compression

Question 9

A data scientist uses SageMaker to train a model and wants to automatically stop the training job if the loss is not improving after a certain number of steps. Which feature should be used?

Accepted Answer

SageMaker Debugger. SageMaker Debugger provides built-in rules that monitor training metrics (e.g., loss) in real time and can trigger actions such as stopping the training job when the loss stops improving for a specified number of steps. This is done via the `StopTrainingJobOnRuleEvaluation` action, which automatically halts the job when a rule like `loss_not_decreasing` is violated.

Answer

SageMaker Experiments

Answer

SageMaker Automatic Model Tuning

Answer

SageMaker Ground Truth

Question 10

Which THREE actions should be taken to ensure data security when training a model using Amazon SageMaker with data stored in Amazon S3? (Choose 3.)

Accepted Answer

Use a VPC to isolate the SageMaker training job. Option A is correct because running the SageMaker training job inside a VPC (with private subnets, security groups, and VPC endpoints for S3 and SageMaker) isolates network traffic and prevents the training data from traversing the public internet. Option D is correct because enabling server-side encryption on the S3 bucket (SSE-S3, SSE-KMS, or SSE-C) protects the training data at rest, so even if objects are accessed improperly the contents remain encrypted. Option E is correct because attaching an IAM role with least-privilege permissions to the SageMaker training job scopes access to only the specific S3 buckets, prefixes, and API actions required, limiting the blast radius of any compromise. Option B is not appropriate because denying all access except from the SageMaker service is overly broad and would break other legitimate consumers of the bucket; access should be controlled via IAM policies and bucket policies scoped to specific principals and actions. Option C is incorrect because training data resides in Amazon S3, not on an attached EBS volume, and SageMaker training instances use ephemeral storage rather than customer-managed EBS volumes for input data.

Answer

Apply an S3 bucket policy that denies all access except from the SageMaker service

Answer

Attach an EBS volume for storing training data

Question 11

Which TWO approaches can reduce inference latency on a SageMaker real-time endpoint? (Choose 2.)

Accepted Answer

Attach an Elastic Inference accelerator. Option A is correct because attaching an Amazon Elastic Inference accelerator to a SageMaker real-time endpoint provides additional low-cost, fractional GPU compute that speeds up deep learning inference, directly reducing per-request latency. Option E is correct because SageMaker Neo compiles the trained model into an optimized executable for the target hardware, which improves inference throughput and lowers latency on the endpoint. Option B is incorrect because increasing batch size typically improves throughput, not latency, and can even raise per-request latency. Option C is incorrect because SageMaker Model Monitor observes data and model quality drift; it does not affect inference speed. Option D is incorrect because using a GPU instance type is a hardware choice that may help, but it is not one of the two approaches specifically identified here as latency-reducing optimizations.

Answer

Increase the batch size

Answer

Enable SageMaker Model Monitor

Answer

Use a GPU instance type

Question 12

A team is deploying a machine learning model to production using Amazon SageMaker. They want to automatically scale the endpoint based on the incoming request volume, and they also need to ensure that the endpoint can handle sudden bursts of traffic without dropping requests. Which scaling policy should they use?

Accepted Answer

Target tracking scaling policy based on the number of invocations. A target tracking scaling policy based on the number of invocations allows the endpoint to automatically adjust the number of instances to maintain a target metric value (e.g., invocations per instance). This policy can proactively scale out to handle sudden bursts by adding instances before the request queue grows, preventing dropped requests. SageMaker's built-in scaling metric, 'SageMakerVariantInvocationsPerInstance', is ideal for this use case as it directly correlates with traffic volume.

Answer

Scheduled scaling policy for peak hours

Answer

Simple scaling policy based on average latency

Answer

Manual scaling by monitoring CloudWatch alarms

Question 13

A company has a real-time inference endpoint on Amazon SageMaker that uses a custom container. The endpoint is experiencing high latency and occasional 502 errors. The logs from the container show that the model inference time is low, but the overall response time is high. Which step is MOST likely to reduce the latency?

Accepted Answer

Increase the number of instances and enable auto-scaling. Increasing the number of instances and enabling auto-scaling helps distribute the incoming request load, reducing queuing delays at the endpoint. The logs show inference time is low, so the high latency is likely due to request queuing behind other requests. Scaling out addresses this. Option A is wrong because batch transform is designed for offline, batch processing, not real-time inference. Option B is wrong because using a larger instance type may provide more compute but does not directly address queuing; it is also less cost-effective than scaling out. Option C is wrong because the model inference time is already low, so further optimization would have minimal impact on overall latency.

Answer

Switch to batch transform to process requests in batches

Answer

Use a larger instance type for the endpoint

Answer

Optimize the model to reduce inference time

Question 14

A machine learning engineer is deploying a model that was trained on a large dataset stored in Amazon S3. The model needs to be retrained daily with new data. Which approach is the MOST cost-effective for storing the training data while allowing quick access for retraining?

Accepted Answer

Use S3 Intelligent-Tiering. S3 Intelligent-Tiering is the most cost-effective choice because it automatically moves data between access tiers (frequent, infrequent, and archive instant access) based on changing access patterns. Since the model is retrained daily, the training data will be accessed frequently during retraining but may have low access at other times, and Intelligent-Tiering optimizes storage costs without performance impact by charging a small monitoring fee per object.

Answer

Store all data in S3 Standard

Answer

Use S3 Glacier Deep Archive

Answer

Use S3 One Zone-IA

Question 15

A company is using Amazon SageMaker to train and deploy machine learning models. The data science team wants to track and compare model versions, hyperparameters, and metrics across multiple training jobs. Which TWO AWS services should they use together to achieve this? (Choose TWO.)

Accepted Answer

Amazon S3. Amazon SageMaker Experiments (option E) is the purpose-built capability within SageMaker for tracking, organizing, and comparing machine learning runs, capturing hyperparameters, metrics, and artifacts across multiple training jobs, so it directly satisfies the requirement to compare model versions and training results. Amazon S3 (option D) is the correct companion service because SageMaker Experiments stores its metadata and the underlying training artifacts, such as model artifacts, datasets, and output files, in S3, making it the durable storage layer for the tracked experiments. Amazon RDS (option A) is a relational database service and is not used by SageMaker Experiments for experiment tracking, so it does not fit the scenario. Amazon CloudWatch Logs (option B) captures log streams and metrics for monitoring and debugging, but it does not provide experiment comparison or hyperparameter tracking. AWS Glue (option C) is a serverless data integration and ETL service, not a model experiment tracking service, so it is unrelated to this requirement.

Answer

Amazon RDS

Answer

Amazon CloudWatch Logs

Answer

AWS Glue