MLS-C01 Machine Learning Implementation and Operations Practice Test 18 — 15 Questions

Question 1

A team is using Amazon SageMaker Autopilot to automatically build models. The dataset has 50 features and 1 million rows. After training, Autopilot generates multiple candidates. The team wants to deploy the model with the highest accuracy. What is the best practice to select and deploy the model?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Deploy the candidate with the highest objective metric value from the Autopilot leaderboard. Amazon SageMaker Autopilot automatically generates a leaderboard ranking candidate models by their objective metric (e.g., accuracy, F1, AUC). The highest-ranked candidate represents the best-performing model based on the validation data, and deploying it directly is the recommended best practice. Autopilot handles preprocessing, algorithm selection, and hyperparameter tuning internally, so manual intervention is unnecessary.

Answer

Deploy all candidates behind a multi-model endpoint and route traffic based on request features

Answer

Select the model with the highest validation accuracy after performing additional hyperparameter tuning

Answer

Manually review each candidate's architecture and select the one with the simplest design

Question 2

An ML engineer needs to store and version training datasets and model artifacts. Which AWS service should they use?

Accepted Answer

Amazon Simple Storage Service (S3). Amazon S3 is the correct choice because it provides scalable, durable, and cost-effective object storage with built-in versioning capabilities, making it ideal for storing and versioning large training datasets and model artifacts. S3's versioning feature allows you to preserve, retrieve, and restore every version of an object, which is essential for reproducibility in ML workflows.

Answer

Amazon DynamoDB

Answer

Amazon Elastic File System (EFS)

Answer

Amazon Elastic Block Store (EBS)

Question 3

A team is training a large language model using PyTorch on multiple GPUs. The training is taking too long due to inefficient data loading. Which AWS service can help accelerate data loading by caching data close to the GPU instances?

Accepted Answer

Amazon FSx for Lustre. Amazon FSx for Lustre is a high-performance file system optimized for machine learning workloads. It provides sub-millisecond latencies and high throughput by caching training data on local NVMe SSDs attached to the Lustre servers, which are co-located with GPU instances in the same AWS Availability Zone. This eliminates the I/O bottleneck from remote object storage, directly accelerating data loading for PyTorch DataLoader workers.

Answer

Amazon EBS Snapshots for fast restore

Answer

Amazon S3 Transfer Acceleration

Answer

Amazon CloudFront

Question 4

A company's ML pipeline uses AWS Step Functions to orchestrate data preprocessing, training, and evaluation. The training step occasionally fails due to a transient error. What is the most robust way to handle this without manual intervention?

Accepted Answer

Implement a retry policy with exponential backoff on the training step in the state machine. AWS Step Functions natively supports retry policies with exponential backoff, which automatically retries failed tasks after a delay that increases progressively. This handles transient errors (e.g., resource contention, network glitches) without manual intervention, making the pipeline robust and self-healing.

Answer

Configure a CloudWatch alarm to notify the team when the step fails

Answer

Use a parallel state to run multiple training instances simultaneously

Answer

Use a custom Lambda function to catch the error and restart the training step

Question 5

A data scientist needs to perform hyperparameter optimization for a gradient boosting model. Which built-in Amazon SageMaker feature should they use?

Accepted Answer

Amazon SageMaker Automatic Model Tuning. Amazon SageMaker Automatic Model Tuning (A) is the built-in feature specifically designed for hyperparameter optimization. It automates the search for the best combination of hyperparameters by launching multiple training jobs with different hyperparameter values, using strategies like Bayesian optimization, random search, or Hyperband. This directly addresses the data scientist's need to optimize a gradient boosting model's hyperparameters.

Answer

Amazon SageMaker Clarify

Answer

Amazon SageMaker Debugger

Answer

Amazon SageMaker Neo

Question 6

A company's ML model is deployed on a SageMaker endpoint. The model's predictions are used in a customer-facing application that requires low latency. Over time, the model's performance degrades due to data drift. What is the most suitable approach to detect this drift automatically?

Accepted Answer

Enable Amazon SageMaker Model Monitor to continuously check for data drift. Amazon SageMaker Model Monitor is purpose-built to automatically detect data drift by continuously comparing incoming inference data against a baseline dataset. It computes statistical metrics (e.g., distribution distances like Kolmogorov-Smirnov or Chi-squared) and raises alerts when drift exceeds configurable thresholds, enabling proactive retraining without manual intervention. This directly addresses the need for automated drift detection in a low-latency customer-facing application.

Answer

Set up a CloudWatch alarm on the endpoint's invocation latency

Answer

Periodically retrain the model using all historical data

Answer

Use Amazon S3 events to trigger a Lambda function that compares distributions

Question 7

An ML team is using SageMaker Processing jobs to run feature engineering scripts. The scripts require a specific Python package not included in the default SageMaker image. How should the team provide this package?

Accepted Answer

Create a custom Docker image that includes the package and use it for the Processing job. SageMaker Processing jobs run in isolated Docker containers, and the default SageMaker images only include pre-installed packages. To add a custom Python package, the team must create a custom Docker image that includes the package (e.g., via a Dockerfile with 'pip install <package>'), then specify that image URI in the Processing job configuration. This ensures the package is available in the container environment before the script executes.

Answer

Include 'pip install <package>' in the processing script

Answer

Use the SageMaker prebuilt deep learning container with the package

Answer

Place a requirements.txt file in the input data S3 bucket

Question 8

Which TWO options are valid ways to reduce inference latency for a model deployed on a SageMaker real-time endpoint? (Select TWO.)

Accepted Answer

Enable SageMaker Neo to compile the model for the target instance. Option C is correct because SageMaker Neo compiles the model for the specific target instance's hardware and instruction set, producing an optimized executable that runs faster and reduces per-invocation inference latency. Option D is correct because GPU instance types (e.g., ml.p3, ml.g4dn) provide massively parallel compute that accelerates the matrix operations in deep learning models, lowering inference latency compared to CPU-only instances. Option A is wrong because batch transform is designed for offline, large-batch scoring jobs, not low-latency real-time inference. Option B is wrong because adding instances behind a load balancer improves throughput and availability, not the latency of a single inference request. Option E is wrong because the invocation timeout only controls how long the client waits before failing; increasing it does not make the model compute any faster.

Answer

Use SageMaker batch transform instead of real-time endpoint

Answer

Deploy the model to multiple instances behind a load balancer

Answer

Increase the endpoint's invocation timeout

Question 9

Which THREE steps should be taken to secure a SageMaker notebook instance that accesses sensitive data? (Select THREE.)

Accepted Answer

Enable encryption at rest for the notebook's EBS volume. Option A is correct because enabling encryption at rest for the notebook's EBS volume protects sensitive data stored on the instance's attached storage using AWS KMS, ensuring data cannot be read if the underlying volume is compromised. Option C is correct because placing the notebook instance inside a VPC with no internet access (i.e., disabling direct internet access and using VPC endpoints or a NAT gateway for required traffic) prevents exfiltration of sensitive data and blocks unauthorized inbound/outbound connections. Option E is correct because attaching an IAM role with least privilege permissions ensures the notebook can only access the specific AWS resources and actions it needs, limiting the blast radius if the instance is compromised. Option B is incorrect because granting root access increases the attack surface and violates the principle of least privilege. Option D is incorrect because allowing direct internet access exposes the notebook to data exfiltration and unauthorized downloads, undermining the security of sensitive data.

Answer

Grant root access to the notebook instance for flexibility

Answer

Allow direct internet access from the notebook for downloading packages

Question 10

Which TWO AWS services can be used to deploy a trained model for serverless inference? (Select TWO.)

Accepted Answer

AWS Lambda with a container image. Option A, AWS Lambda with a container image, is correct because Lambda is a fully serverless compute service that can run inference code packaged as a container image (up to 10 GB) without managing servers, scaling automatically per request. Option B, Amazon SageMaker Serverless Inference, is correct because it is purpose-built to host trained models with automatic scaling and scale-to-zero, charging only for compute used during inference without provisioning instances. Option C, SageMaker batch transform, is not serverless inference for real-time requests; it is a batch job that provisions resources to process an entire dataset offline. Option D, ECS with Fargate, is serverless for containers but is not a model-serving inference service and requires managing task definitions and services. Option E, EC2 instances, requires provisioning and managing servers, so it is not serverless.

Answer

Amazon SageMaker batch transform

Answer

Amazon Elastic Container Service (ECS) with Fargate

Answer

Amazon EC2 instances

Question 11

A company is running a real-time inference endpoint on Amazon SageMaker. The endpoint is using an ml.c5.xlarge instance. Over the past month, the CPU utilization has been consistently below 10%, and the latency is well within requirements. The company wants to reduce costs. What should they do?

Accepted Answer

Use a smaller instance type. The CPU utilization is consistently below 10%, indicating significant over-provisioning. Downgrading to a smaller instance type (e.g., ml.c5.large or ml.t3.medium) directly reduces the per-hour cost while still meeting the latency requirements. This is the most straightforward cost optimization when the current instance is underutilized and performance is already satisfactory.

Answer

Set up a scaling policy to scale down to zero

Answer

Switch to a multi-model endpoint

Answer

Use a batch transform job instead

Answer

Move to a serverless inference endpoint

Question 12

A data scientist is using Amazon SageMaker to train a model. The training job is taking longer than expected. The data scientist notices that the GPU utilization is low. Which action would most likely improve GPU utilization?

Accepted Answer

Increase the batch size. Low GPU utilization during training often indicates that the GPU is waiting for data to process, a condition known as data bottleneck. Increasing the batch size allows the GPU to process more samples per forward/backward pass, keeping it busier and improving utilization. In SageMaker, this directly impacts the training loop by reducing the frequency of data loading and model update steps.

Answer

Change to a CPU-based instance

Answer

Decrease the batch size

Answer

Use a larger instance type

Answer

Enable data augmentation

Question 13

A company is using Amazon SageMaker to deploy a model. The model is a large ensemble that requires 8 GB of memory. The company wants to minimize endpoint cost. Which instance type should they choose?

Accepted Answer

ml.m5.large. The ml.m5.large instance provides 8 GB of memory (2 vCPUs, 8 GiB RAM), which meets the model's requirement of 8 GB. It is typically priced lower than ml.c5.xlarge in AWS (e.g., $0.115/hr vs $0.17/hr), making it the most cost-effective choice among instances that can accommodate the memory. The ml.c5.xlarge also provides 8 GB but at a higher cost, while smaller instances lack sufficient memory and larger instances incur unnecessary expense.

Answer

ml.c5.xlarge

Answer

ml.t2.medium

Answer

ml.c5.large

Answer

ml.c5.2xlarge

Question 14

A company is using Amazon SageMaker to train a model. The training data includes sensitive personally identifiable information (PII). The company needs to ensure that the training data is protected and that the trained model does not inadvertently expose PII. Which TWO actions should the company take? (Choose TWO.)

Accepted Answer

Encrypt the training data in S3 using AWS KMS. Option A is correct because encrypting the training data in Amazon S3 with AWS KMS (SSE-KMS) protects the PII at rest, ensuring that even if storage is compromised the data remains unreadable without the appropriate KMS key permissions. Option C is correct because using SageMaker's data processing (e.g., SageMaker Processing jobs with PII detection/redaction logic) to redact or anonymize PII before training prevents the model from learning and later exposing sensitive information. Option B is not sufficient because SSE-S3 uses S3-managed keys, which offer less control, auditing, and customer-managed key separation than AWS KMS for sensitive PII. Option D does not protect data or prevent PII exposure; CloudTrail only provides audit logging of API activity. Option E is wrong because granting public read access would directly expose the PII and violate the protection requirement.

Answer

Use server-side encryption with S3-managed keys

Answer

Enable AWS CloudTrail to log all access to the data

Answer

Grant public read access to the training data for faster access

Question 15

A data scientist is deploying a model on Amazon SageMaker. The model requires inference on images, and the data scientist wants to use a GPU instance for low latency. However, the data scientist is unsure about the instance type to choose for the endpoint. Which TWO factors should the data scientist consider when selecting the instance type? (Choose TWO.)

Accepted Answer

The cost per inference for the instance type. When selecting an instance type for a SageMaker endpoint, the data scientist should consider the cost per inference for the instance type (option C), because endpoint instances are billed continuously while running, so the instance's hourly price directly determines the cost efficiency of serving predictions, especially for low-latency GPU workloads. The data scientist should also consider the GPU memory available on the instance (option E), since image inference models (e.g., CNN-based models) must fit their model weights and intermediate tensors into GPU memory; insufficient GPU memory causes out-of-memory errors or forces smaller batch sizes that hurt latency. Option A is not relevant because training time is determined during the training job, not by the inference endpoint instance type. Option B is not the key factor here because the workload is GPU-based, so GPU memory and GPU compute matter more than vCPU count for low-latency image inference. Option D is irrelevant because the AWS Region of the S3 bucket storing the model does not determine the endpoint instance type; the model artifacts are downloaded to the endpoint instance regardless of bucket region.

Answer

The time taken to train the model

Answer

The number of vCPUs on the instance

Answer

The AWS Region of the S3 bucket storing the model