MLS-C01 Machine Learning Implementation and Operations Practice Test 3 — 15 Questions

Question 1

A media company uses Amazon SageMaker to train a deep learning model for video classification. The training job uses a single ml.p3.2xlarge instance and processes 50 GB of labeled video data stored in Amazon S3. The training completes successfully in 12 hours. However, the data scientists report that the model’s accuracy is lower than expected. They suspect the training data contains labeling errors. To improve model accuracy without incurring significant additional cost, they want to identify and remove mislabeled training examples before retraining. They have a small budget of $50 and need to complete the analysis within 2 hours. Which approach should the data scientists take?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Train a small, fast model on a random sample of the data (e.g., 1 GB) using a cheaper instance like ml.m5.xlarge, then use the model's prediction confidence to flag low-confidence examples as potential mislabels for manual review.. Training a small, fast model on a 1 GB random sample using a cheaper instance (ml.m5.xlarge) allows the team to quickly identify low-confidence predictions, which are strong indicators of mislabeled examples. This approach fits within the $50 budget and 2-hour time constraint, as it avoids processing the full 50 GB dataset and leverages a lightweight model for rapid iteration. By flagging only suspicious samples for manual review, the team can efficiently clean the training data without incurring the cost of re-labeling the entire dataset.

Answer

Use SageMaker Ground Truth to create a new labeling job for the entire dataset, then compare the new labels with the original labels to identify discrepancies.

Answer

Use SageMaker Clarify to generate a bias report for the training data and remove instances that contribute to bias.

Answer

Manually review all 50 GB of video data to correct labels.

Question 2

A company wants to deploy a machine learning model that performs real-time inference with sub-second latency. The model is a deep neural network with 500 MB of weights. The inference endpoint must scale to zero when not in use to minimize cost. Which AWS service should the company use?

Accepted Answer

Use Amazon SageMaker Serverless Inference to host the model.. Amazon SageMaker Serverless Inference is designed for workloads with intermittent traffic patterns, automatically scaling to zero when idle and scaling up for real-time requests. It supports models up to 1 GB in size and provides sub-second latency for inference, making it ideal for this 500 MB deep neural network. This service eliminates the need to manage underlying infrastructure while meeting the latency and cost requirements.

Answer

Deploy the model as an AWS Lambda function with provisioned concurrency.

Answer

Host the model on Amazon ECS with Fargate and use a target tracking scaling policy.

Answer

Create an Amazon SageMaker real-time endpoint with automatic scaling policies.

Question 3

A data science team is training a large deep learning model using Amazon SageMaker. The training job is taking a long time because the model has many layers and the dataset is large. The team wants to reduce training time by distributing the training across multiple GPUs on a single instance, as well as across multiple instances. Which TWO actions should the team take? (Choose two.)

Accepted Answer

Configure the training job to use SageMaker's model parallelism (SMP) library for pipeline or tensor parallelism.. The SageMaker model parallelism (SMP) library is specifically designed to split large deep learning models across multiple GPUs using pipeline or tensor parallelism. This allows the team to train models that are too large to fit on a single GPU and to reduce training time by parallelizing computation across devices within and across instances.

Answer

Use SageMaker's distributed data parallelism (SMDDP) library to shard the model across GPUs.

Answer

Use SageMaker's managed training with a single instance containing multiple GPUs and enable data parallelism.

Answer

Set the instance type to a single GPU instance and rely on automatic model parallelism.

Question 4

An ML engineer is troubleshooting why an automated CI/CD pipeline cannot deploy an updated model to an existing SageMaker endpoint. The pipeline uses the IAM role that has the attached policy shown in the exhibit. What is the MOST likely cause of the failure?

Accepted Answer

The pipeline tries to update an existing endpoint, but the sagemaker:UpdateEndpoint action is not allowed.. The pipeline is attempting to deploy an updated model to an existing SageMaker endpoint, which requires the sagemaker:UpdateEndpoint action. The IAM policy shown in the exhibit (not provided here but implied) does not include this action, so the API call fails with an access denied error. Without explicit permission to update the endpoint, the CI/CD pipeline cannot modify the deployed configuration.

Answer

The pipeline tries to create a new endpoint, but the sagemaker:CreateEndpoint action is denied.

Answer

The pipeline tries to delete the old endpoint, but the sagemaker:DeleteEndpoint action is denied by a Deny statement.

Answer

The pipeline attempts to invoke the endpoint, but the sagemaker:InvokeEndpoint action is denied.

Question 5

Drag and drop the steps to train a model using Amazon SageMaker built-in algorithm in the correct order.

Accepted Answer

Prepare training data and upload to S3 → Choose built-in algorithm → Create training job specifying algorithm, input/output S3 paths, and compute resources → Execute training job. Training involves data preparation, job creation, algorithm selection, input/output paths, and execution.

Answer

Choose built-in algorithm → Prepare training data and upload to S3 → Create training job specifying algorithm, input/output S3 paths, and compute resources → Execute training job

Answer

Prepare training data and upload to S3 → Create training job specifying input/output S3 paths and compute resources → Choose built-in algorithm → Execute training job

Answer

Create training job specifying algorithm, input/output S3 paths, and compute resources → Prepare training data and upload to S3 → Execute training job → Choose built-in algorithm

Question 6

Match each SageMaker built-in algorithm to its primary use case.

Accepted Answer

Linear Learner: Used for regression and binary/multiclass classification.. Correct matches: Linear Learner → regression/classification, Object Detection → image object detection. Common confusions: swapping XGBoost (gradient boosting) with BlazingText (text vectors).

Answer

XGBoost: Used for word2vec and text classification.

Answer

BlazingText: Gradient boosting algorithm for classification and regression.

Question 7

Match each SageMaker built-in metric to its meaning.

Accepted Answer

accuracy: The ratio of correctly predicted observations to total observations. Accuracy measures overall correctness, precision measures positive predictive value, and recall measures sensitivity. Common confusions occur when precision and recall definitions are swapped.

Answer

accuracy: The ratio of true positives to the sum of true positives and false positives

Answer

precision: The ratio of true positives to the sum of true positives and false negatives

Answer

recall: The ratio of correctly predicted observations to total observations

Question 8

A data scientist is training a linear regression model on a dataset with 100 features. The model shows high variance on the test set. Which action is MOST likely to reduce overfitting?

Accepted Answer

Apply L2 regularization (Ridge regression). High variance (overfitting) means the model is too complex and fits noise in the training data. L2 regularization (Ridge regression) adds a penalty proportional to the square of the coefficients, shrinking them toward zero and reducing model complexity. This directly counteracts overfitting by preventing the model from relying too heavily on any single feature.

Answer

Use a more complex model like XGBoost

Answer

Increase the number of training iterations

Answer

Add more feature engineering to increase model complexity

Question 9

A company is using Amazon SageMaker to deploy a real-time inference endpoint for a computer vision model. The endpoint receives bursts of traffic with up to 500 requests per second, but the load is unpredictable. Which scaling strategy is MOST cost-effective while maintaining low latency?

Accepted Answer

Configure automatic scaling with a target tracking policy and add a buffer to handle bursts. Amazon SageMaker's automatic scaling with a target tracking policy dynamically adjusts the number of instances based on a target metric (e.g., InvocationsPerInstance), which handles unpredictable bursts cost-effectively. Adding a buffer (e.g., a higher target value or a cooldown period) ensures low latency by pre-scaling before traffic spikes, avoiding cold starts and over-provisioning.

Answer

Manually provision enough instances to handle peak load

Answer

Use provisioned concurrency on SageMaker Serverless Inference

Answer

Use a multi-model endpoint to reduce the number of instances

Question 10

A machine learning team is building a fraud detection system using Amazon SageMaker. The training data is highly imbalanced (99% legitimate, 1% fraudulent). They need to maximize the recall of the fraud class while keeping precision above 90%. Which approach should they take?

Accepted Answer

Train a model using the original data, then adjust the decision threshold on the validation set to maximize recall while precision > 90%. Adjusting the decision threshold on the validation set directly optimizes the trade-off between recall and precision. By lowering the threshold, the model classifies more instances as fraud, increasing recall, while the precision constraint (≥90%) ensures the threshold is set at a point where false positives remain acceptably low. This approach works with any probabilistic classifier and does not alter the training data distribution.

Answer

Undersample the majority class to create a balanced dataset and train a Random Forest

Answer

Train an XGBoost model with scale_pos_weight parameter set to 99

Answer

Use SMOTE to oversample the fraud class and then train a logistic regression

Question 11

A data scientist wants to use Amazon SageMaker to train a deep learning model on a large dataset stored in S3. The training job is expected to take several hours. Which storage option should be used to minimize data loading time and cost?

Accepted Answer

Use Pipe mode to stream data directly from S3 during training. Pipe mode is the correct choice because it streams data directly from S3 into the training algorithm without writing to disk, eliminating the time and cost of copying large datasets to the instance's local storage. This minimizes data loading time (streaming starts immediately) and cost (no EBS volume or additional storage charges), making it ideal for large datasets that take hours to train.

Answer

Attach an Amazon EBS volume with the dataset pre-loaded

Answer

Use File mode to copy data to the training instance's local storage

Answer

Mount an Amazon EFS file system to the training instance

Question 12

An ML engineer is deploying a model to a SageMaker endpoint for real-time inference. The model requires a custom inference script that preprocesses input data and postprocesses predictions. Which SageMaker feature should be used to implement this custom logic?

Accepted Answer

Create a SageMaker model with a custom inference script that includes pre- and post-processing functions. SageMaker allows you to bring your own container or use a pre-built container with a custom inference script that defines `input_fn`, `predict_fn`, `output_fn`, and `model_fn` functions. These functions handle preprocessing of input data, model prediction, and postprocessing of predictions, enabling custom logic for real-time inference endpoints without requiring separate infrastructure.

Answer

Use SageMaker Ground Truth to transform inference requests

Answer

Use SageMaker Processing jobs to preprocess data before inference

Answer

Use a built-in SageMaker algorithm with the default inference code

Question 13

A company uses Amazon SageMaker to train a text classification model. The training data is stored in S3 and contains sensitive personally identifiable information (PII). The company must ensure that the data is encrypted at rest in S3 and that the encryption key is managed by the company's own hardware security module (HSM). Which configuration should be used?

Accepted Answer

Use S3 server-side encryption with customer-provided keys (SSE-C) and store the keys in the HSM. SSE-C allows you to provide your own encryption key, which can be stored in your own HSM, and Amazon S3 manages the encryption/decryption process using that key. This satisfies the requirement for server-side encryption with a key managed by the company's HSM, ensuring data at rest is encrypted without exposing the key to AWS.

Answer

Use S3 server-side encryption with S3-managed keys (SSE-S3)

Answer

Use client-side encryption with the encryption key stored in the HSM

Answer

Use S3 server-side encryption with AWS KMS managed keys (SSE-KMS) with a customer managed key

Question 14

A data scientist is using Amazon SageMaker to train an XGBoost model on a dataset with missing values. The dataset has both numeric and categorical features. Which preprocessing step is MOST appropriate before training?

Accepted Answer

One-hot encode categorical features and let XGBoost handle missing values natively. XGBoost has a built-in mechanism to handle missing values by learning the best direction to split on during training, making explicit imputation unnecessary. One-hot encoding categorical features is required because XGBoost only accepts numeric inputs, and this encoding preserves the categorical information without imposing ordinal relationships. This approach avoids data leakage from imputation and leverages XGBoost's native sparsity-aware algorithm.

Answer

Impute missing numeric values with the mean and categorical values with the mode, then train without encoding

Answer

Remove all rows with missing values and train on the remaining data

Answer

Label encode categorical features and use the built-in missing value handling of XGBoost

Question 15

A company is using Amazon SageMaker to host a model that performs real-time inference. The model receives around 100 requests per second with occasional spikes up to 500 requests per second. The current endpoint uses 2 ml.m5.large instances. During spikes, latency increases significantly, and some requests time out. What is the MOST cost-effective solution to handle the spikes without losing requests?

Accepted Answer

Configure SageMaker managed scaling with a target tracking policy and add a buffer based on the average spike duration. SageMaker managed scaling with a target tracking policy automatically adjusts the number of instances based on a specified metric (e.g., invocation count or latency), and adding a buffer based on the average spike duration ensures that additional capacity is provisioned before the spike hits, preventing timeouts. This is the most cost-effective approach as it scales out during spikes and scales in during normal load, avoiding over-provisioning.

Answer

Replace the instances with a single larger instance type, such as ml.m5.4xlarge

Answer

Use an Amazon SQS queue to buffer incoming requests and process them asynchronously

Answer

Use AWS Lambda with a provisioned concurrency to handle the spikes