MLS-C01 Machine Learning Implementation and Operations Practice Test 2 — 15 Questions

Question 1

Match each SageMaker built-in metric to its meaning.

Accepted Answer

accuracy: The ratio of correctly predicted observations to total observations. Accuracy measures overall correctness, precision measures positive predictive value, and recall measures sensitivity. Common confusions occur when precision and recall definitions are swapped.

Answer

accuracy: The ratio of true positives to the sum of true positives and false positives

Answer

precision: The ratio of true positives to the sum of true positives and false negatives

Answer

recall: The ratio of correctly predicted observations to total observations

Question 2

A company is using Amazon SageMaker to deploy a real-time inference endpoint for a computer vision model. The endpoint receives bursts of traffic with up to 500 requests per second, but the load is unpredictable. Which scaling strategy is MOST cost-effective while maintaining low latency?

Accepted Answer

Configure automatic scaling with a target tracking policy and add a buffer to handle bursts. Amazon SageMaker's automatic scaling with a target tracking policy dynamically adjusts the number of instances based on a target metric (e.g., InvocationsPerInstance), which handles unpredictable bursts cost-effectively. Adding a buffer (e.g., a higher target value or a cooldown period) ensures low latency by pre-scaling before traffic spikes, avoiding cold starts and over-provisioning.

Answer

Manually provision enough instances to handle peak load

Answer

Use provisioned concurrency on SageMaker Serverless Inference

Answer

Use a multi-model endpoint to reduce the number of instances

Question 3

A company uses Amazon SageMaker to train a text classification model. The training data is stored in S3 and contains sensitive personally identifiable information (PII). The company must ensure that the data is encrypted at rest in S3 and that the encryption key is managed by the company's own hardware security module (HSM). Which configuration should be used?

Accepted Answer

Use S3 server-side encryption with customer-provided keys (SSE-C) and store the keys in the HSM. SSE-C allows you to provide your own encryption key, which can be stored in your own HSM, and Amazon S3 manages the encryption/decryption process using that key. This satisfies the requirement for server-side encryption with a key managed by the company's HSM, ensuring data at rest is encrypted without exposing the key to AWS.

Answer

Use S3 server-side encryption with S3-managed keys (SSE-S3)

Answer

Use client-side encryption with the encryption key stored in the HSM

Answer

Use S3 server-side encryption with AWS KMS managed keys (SSE-KMS) with a customer managed key

Question 4

A data scientist is using Amazon SageMaker to train an XGBoost model on a dataset with missing values. The dataset has both numeric and categorical features. Which preprocessing step is MOST appropriate before training?

Accepted Answer

One-hot encode categorical features and let XGBoost handle missing values natively. XGBoost has a built-in mechanism to handle missing values by learning the best direction to split on during training, making explicit imputation unnecessary. One-hot encoding categorical features is required because XGBoost only accepts numeric inputs, and this encoding preserves the categorical information without imposing ordinal relationships. This approach avoids data leakage from imputation and leverages XGBoost's native sparsity-aware algorithm.

Answer

Impute missing numeric values with the mean and categorical values with the mode, then train without encoding

Answer

Remove all rows with missing values and train on the remaining data

Answer

Label encode categorical features and use the built-in missing value handling of XGBoost

Question 5

An ML team is using Amazon SageMaker to train a model. They notice that the training job is taking longer than expected and the CloudWatch metrics show high GPU utilization but low CPU utilization. Which action is MOST likely to improve training speed?

Accepted Answer

Use SageMaker Pipe mode to stream data from S3 to reduce I/O bottleneck. High GPU utilization with low CPU utilization typically indicates that the GPU is actively computing and the CPU is not a bottleneck for data preprocessing. However, this does not rule out I/O inefficiencies at the storage layer. SageMaker Pipe mode streaming eliminates the need to download data to the local filesystem first, reducing I/O wait times. Even when the GPU is busy, reducing I/O overhead can speed up the training by decreasing the time between batches and allowing the GPU to process more data per unit time.

Answer

Switch to a CPU-only instance to avoid GPU overhead

Answer

Increase the number of training instances

Answer

Use a larger GPU instance with more GPU memory

Question 6

A company is using Amazon SageMaker to host a model that performs real-time inference. The model receives around 100 requests per second with occasional spikes up to 500 requests per second. The current endpoint uses 2 ml.m5.large instances. During spikes, latency increases significantly, and some requests time out. What is the MOST cost-effective solution to handle the spikes without losing requests?

Accepted Answer

Configure SageMaker managed scaling with a target tracking policy and add a buffer based on the average spike duration. SageMaker managed scaling with a target tracking policy automatically adjusts the number of instances based on a specified metric (e.g., invocation count or latency), and adding a buffer based on the average spike duration ensures that additional capacity is provisioned before the spike hits, preventing timeouts. This is the most cost-effective approach as it scales out during spikes and scales in during normal load, avoiding over-provisioning.

Answer

Replace the instances with a single larger instance type, such as ml.m5.4xlarge

Answer

Use an Amazon SQS queue to buffer incoming requests and process them asynchronously

Answer

Use AWS Lambda with a provisioned concurrency to handle the spikes

Question 7

Which TWO configuration steps are necessary to deploy a custom Docker container for training in Amazon SageMaker? (Choose two.)

Accepted Answer

Implement the train function in the container that saves model artifacts to /opt/ml/model. Option B is correct because SageMaker training containers must implement the training logic and write the resulting model artifacts to the reserved local path /opt/ml/model, which SageMaker then packages and uploads to Amazon S3 after the job completes. Option D is correct because SageMaker passes hyperparameters to the container in the file /opt/ml/input/config/hyperparameters.json, so the training script must read them from that exact location to honor the values supplied in the CreateTrainingJob request. Option A is not required because a REST inference endpoint pertains to hosting/deployment, not to a training container. Option C is not required because SageMaker runs a single training container per job and does not use Docker Compose for multi-container orchestration. Option E is not required because the image can be stored in Amazon ECR (the typical choice) or any registry the execution role can pull from, so Docker Hub is not mandatory.

Answer

Expose a REST API endpoint for inference

Answer

Define a Docker Compose file to manage multi-container training

Answer

Push the container image to Docker Hub

Question 8

Which THREE actions can help reduce the inference latency of a SageMaker endpoint? (Choose three.)

Accepted Answer

Use a larger instance type with more CPU/GPU. Using a larger instance type with more CPU or GPU resources directly increases the computational capacity available for inference. This reduces the time required to process each prediction request, thereby lowering inference latency. SageMaker endpoints scale horizontally and vertically, and vertical scaling (larger instances) is a straightforward way to improve per-request performance.

Answer

Use a multi-model endpoint to share instances across models

Answer

Increase the number of instances in the endpoint

Question 9

Which TWO services can be used to perform hyperparameter tuning in Amazon SageMaker? (Choose two.)

Accepted Answer

Amazon SageMaker Automatic Model Tuning. Amazon SageMaker Automatic Model Tuning (option A) is correct because it is the dedicated SageMaker feature that automatically searches for the best hyperparameter configuration by running multiple training jobs against a specified objective metric, using strategies such as Bayesian optimization, random search, and hyperband. Amazon SageMaker Experiments (option B) is also correct because it provides experiment tracking that lets you create, manage, analyze, and compare trials — including runs with different hyperparameter values — so you can organize and evaluate tuning results across training jobs. AWS Glue (option C) is a serverless ETL and data catalog service, not a hyperparameter tuning tool. Amazon SageMaker Ground Truth (option D) is a data labeling service for building training datasets, and Amazon EMR (option E) is a managed big-data processing framework (Hadoop/Spark), neither of which performs hyperparameter tuning.

Answer

AWS Glue

Answer

Amazon SageMaker Ground Truth

Answer

Amazon EMR

Question 10

An IAM policy is attached to a SageMaker notebook instance. The data scientist wants to use the notebook to train a model using data from S3 bucket 'my-bucket'. However, the training job fails with an access denied error. What is the MOST likely cause?

Accepted Answer

The notebook instance role does not have iam:PassRole permission to pass the SageMaker execution role. The most likely cause is that the notebook instance role lacks the iam:PassRole permission, which is required to pass the SageMaker execution role to the training job. When a SageMaker notebook instance creates a training job, it must pass an execution role that the training job will assume to access resources like S3. Without iam:PassRole on the notebook's role, the API call fails with an access denied error, even if all other permissions are correctly configured.

Answer

The sagemaker:CreateTrainingJob permission is not allowed on the specific resource

Answer

The S3 bucket resource ARN is incorrectly formatted

Answer

The s3:GetObject permission is missing for the bucket

Question 11

A SageMaker endpoint creation fails with the above CloudWatch Logs excerpt. What is the MOST likely cause?

Accepted Answer

The model file is too large and takes longer than 300 seconds to load. The CloudWatch Logs excerpt shows a timeout error during model loading. SageMaker has a default 300-second timeout for downloading and loading model artifacts from S3 into the inference container. If the model file is too large or the network is slow, the container fails to start within this window, causing the endpoint creation to fail with a timeout error.

Answer

The S3 bucket containing the model artifacts has incorrect permissions

Answer

The inference script has a syntax error

Answer

The instance type does not have enough memory to load the model

Question 12

A data scientist runs the AWS CLI command shown in the exhibit. The output shows that job-2 failed. Which action should the data scientist take to diagnose the failure?

Accepted Answer

Run `aws sagemaker describe-training-job --training-job-name job-2` to see the failure reason. The `describe-training-job` API call returns a `FailureReason` field that provides the specific error message for a failed SageMaker training job. This is the most direct and efficient way to diagnose why job-2 failed, as it retrieves the exact failure reason from the SageMaker service without requiring additional log parsing or bucket inspection.

Answer

Check the CloudWatch Logs log group for job-2

Answer

Check the S3 bucket for any error logs uploaded by the training job

Answer

Run `aws sagemaker list-training-jobs --name-contains job-2` to get more details

Question 13

You are deploying a custom Docker container for a SageMaker model that requires a specific NVIDIA CUDA version. Which THREE steps must you take to ensure the container runs correctly on SageMaker?

Accepted Answer

Include the SageMaker inference toolkit in the container. Option C is correct because the SageMaker inference toolkit (sagemaker-inference) provides the required HTTP server and model-loading handlers that let a custom container respond to SageMaker's InvokeEndpoint calls. Option D is correct because CUDA requires NVIDIA GPU hardware, so the endpoint must be deployed on a GPU instance type (e.g., ml.p3, ml.g4dn, ml.g5) to expose the driver and GPU to the container. Option E is correct because the container image must define an entry point (or CMD) that launches the inference code/server so SageMaker can start the model when the endpoint is created. Option A is not required because SageMaker manages the endpoint health check through its own ping/health API rather than a user-defined health check endpoint. Option B is not required because Batch Transform is a separate batch-inference feature and is unrelated to ensuring CUDA compatibility for a real-time endpoint.

Answer

Define a health check endpoint

Answer

Use SageMaker Batch Transform

Question 14

A machine learning pipeline uses SageMaker Processing jobs for feature engineering. Which TWO are benefits of using SageMaker Processing over running a custom script on an EC2 instance?

Accepted Answer

Automatically manages the compute resources. Option A is correct because SageMaker Processing is a fully managed service that automatically provisions, scales, and tears down the underlying compute instances for the job, so you don't have to launch or manage an EC2 instance yourself. Option B is correct because SageMaker Processing jobs natively integrate with SageMaker Experiments, automatically logging parameters, metrics, and artifacts so runs can be tracked and compared without custom instrumentation. Option C is not correct because SageMaker does not provide a built-in VPC; you must configure the job to run in your own VPC/subnets for network isolation. Option D is not correct because although Processing supports custom containers, they must be stored in Amazon ECR, not "any registry." Option E is not correct because Processing does not inherently support multiple programming languages beyond what you package in your container; this is not a distinguishing benefit over an EC2 script.

Answer

Provides a built-in VPC for network isolation

Answer

Allows use of custom Docker images from any registry

Answer

Supports multiple programming languages

Question 15

Refer to the exhibit. A SageMaker endpoint is returning 5xx errors. The logs show the above error. Which change will most likely resolve the issue?

Accepted Answer

Use a larger instance type with more memory. 5xx errors from a SageMaker endpoint typically indicate that the inference container is running out of memory (OOM) or crashing under load. The error log suggests the model or inference process requires more memory than the current instance type provides. Upgrading to a larger instance type with more memory directly addresses the resource exhaustion, allowing the model to load and inference to complete without failure.

Answer

Reduce the batch size in the inference script

Answer

Enable Auto Scaling on the endpoint

Answer

Compress the model artifact