MLA-C01 ML Model Development Practice Test 5 — 15 Questions

Question 1

A machine learning engineer is preparing a training job on SageMaker with a custom Docker container. Which TWO actions are required to use the container with SageMaker? (Choose TWO.)

Accepted Answer

Push the container image to Amazon ECR. To use a custom container, you must push it to Amazon ECR and specify the registry path in the estimator. The container must also implement the SageMaker training contract (like /opt/ml), but that is part of building the image.

Answer

Upload the container image to Amazon S3

Answer

Enable SageMaker Debugger to monitor the custom container

Answer

Register the container in SageMaker Model Registry

Question 2

A data scientist is using SageMaker Experiments to track multiple training runs. They want to compare runs based on the objective metric and visualize performance. Which THREE steps should they perform? (Choose THREE.)

Accepted Answer

Use SageMaker Studio Experiments UI to list and compare trials. To track and compare runs, you create an experiment, log parameters and metrics, and then use the Experiments UI or SDK to list and compare trials.

Answer

Deploy the best model to an endpoint

Answer

Enable SageMaker Model Monitor for each run

Question 3

A company wants to use SageMaker Clarify to analyze bias in their training data and model predictions. Which TWO types of bias can Clarify detect? (Choose TWO.)

Accepted Answer

Pre-training bias. SageMaker Clarify can detect pre-training bias (in the data) and post-training bias (in the model predictions).

Answer

Algorithmic bias

Answer

Inference bias

Answer

Deployment bias

Question 4

A data scientist is training an XGBoost model on a large dataset using a SageMaker Training Job. They want to minimize costs without sacrificing model performance. Which instance type and training strategy should they choose?

Accepted Answer

Use a single ml.g4dn.xlarge Spot instance with no distributed training. Using Spot instances with Managed Spot Training can reduce costs by up to 90% compared to On-Demand, and SageMaker automatically handles interruptions. For single-instance training, a single ml.g4dn.xlarge provides sufficient compute for moderate-sized datasets.

Answer

Use a single ml.m5.large On-Demand instance with model parallelism

Answer

Use multiple ml.trn1.2xlarge On-Demand instances with data parallelism

Answer

Use a single ml.p3.2xlarge On-Demand instance with data parallelism

Question 5

A team is fine-tuning a large language model (LLM) using SageMaker and wants to reduce memory footprint during training. Which technique should they use?

Accepted Answer

Use QLoRA (Quantized Low-Rank Adaptation) with 4-bit quantization. QLoRA (Quantized Low-Rank Adaptation) combines 4-bit quantization with low-rank adapters, significantly reducing GPU memory usage while maintaining model quality.

Answer

Use LoRA (Low-Rank Adaptation) with fp32 precision

Answer

Use SageMaker Model Parallelism with tensor parallelism

Answer

Full fine-tuning on a p3.16xlarge instance

Question 6

A machine learning engineer wants to automatically track hyperparameters, metrics, and artifacts for multiple training runs. Which SageMaker feature should they use?

Accepted Answer

SageMaker Experiments. SageMaker Experiments is purpose-built for tracking and comparing training runs, capturing parameters, metrics, and artifacts.

Answer

SageMaker Debugger

Answer

SageMaker Model Monitor

Answer

SageMaker Clarify

Question 7

A data scientist uses SageMaker Automatic Model Tuning (AMT) with Bayesian optimization to tune an XGBoost model. The objective metric is validation:auc, but the tuning job converges to a plateau early. Which action is MOST effective to improve exploration?

Accepted Answer

Increase the exploration_weight parameter in the tuning configuration. Increasing the exploration/exploitation weight (exploration_weight) in Bayesian optimization encourages the algorithm to try more diverse hyperparameter combinations, avoiding premature convergence.

Answer

Increase the number of max parallel jobs

Answer

Decrease the number of hyperparameters being tuned

Answer

Switch the tuning strategy from Bayesian to Random Search

Question 8

A team is training a PyTorch model using SageMaker and wants to use their own custom training container with a specific PyTorch version. Which approach should they use?

Accepted Answer

Use SageMaker Bring Your Own Container (BYOC) with a custom Docker image. BYOC (Bring Your Own Container) allows teams to package their own environment, including custom PyTorch versions, into a Docker container and use it with SageMaker.

Answer

Use the SageMaker built-in PyTorch estimator and set the framework_version

Answer

Use SageMaker Script Mode with a PyTorch script

Answer

Use SageMaker Autopilot to automatically select the container

Question 9

A data scientist trains a binary classification model using SageMaker and obtains an AUC of 0.95 on the test set. However, the precision-recall curve shows low precision for high recall thresholds. The business requires a model that performs well on the minority class. Which metric should the team primarily optimize during hyperparameter tuning?

Accepted Answer

F1-score on the validation set. For imbalanced datasets, the F1-score balances precision and recall, making it a better objective than AUC, which can be misleading when class imbalance exists.

Answer

Accuracy

Answer

AUC (Area Under the ROC Curve)

Answer

Log loss

Question 10

Which SageMaker built-in algorithm should be used for forecasting time series data with seasonal patterns?

Accepted Answer

DeepAR. DeepAR is a supervised learning algorithm for time series forecasting that handles seasonality and trends.

Answer

IP Insights

Answer

BlazingText

Answer

Factorization Machines

Question 11

A data scientist suspects that a deep learning model is overfitting. They enable SageMaker Debugger and want to detect overfitting automatically. Which built-in rule should they use?

Accepted Answer

Overfit. The overfit rule in SageMaker Debugger monitors training and validation loss divergence, a key indicator of overfitting.

Answer

ExplodingGradients

Answer

PoorWeightInitialization

Answer

DeadRelu

Question 12

A company uses SageMaker Autopilot to build a binary classification model. The generated leaderboard shows an ensemble model as the best candidate. The team needs a model that can be deployed for real-time inference with latency < 10ms. What should they do?

Accepted Answer

Select the best single model from the leaderboard (non-ensemble candidate) and deploy it. Autopilot ensemble models may have high latency due to combining multiple models. The team should evaluate non-ensemble candidates (single model) from the leaderboard to meet latency requirements.

Answer

Use SageMaker Inference Recommender to profile the ensemble model and optimize it

Answer

Deploy the ensemble model as a SageMaker endpoint; ensemble models are optimized for low latency

Answer

Retrain the ensemble model with fewer base estimators using a custom container

Question 13

A data scientist is training a linear learner model using SageMaker and notices that the loss is not decreasing. They suspect the issue is exploding gradients. Which SageMaker Debugger rule should they enable to monitor this?

Accepted Answer

ExplodingGradients. The ExplodingGradients rule tracks gradient values and alerts if they exceed a threshold, which is the correct detection for exploding gradients.

Answer

LossNotDecreasing

Answer

VanishingGradients

Answer

Overfit

Question 14

Which SageMaker feature provides AutoML capabilities, including automatic data preprocessing, model selection, and hyperparameter tuning?

Accepted Answer

SageMaker Autopilot. SageMaker Autopilot automates the ML pipeline from data to model, including preprocessing, algorithm selection, and tuning.

Answer

SageMaker Data Wrangler

Answer

SageMaker Automatic Model Tuning

Answer

SageMaker Experiments

Question 15

A machine learning engineer is training a TensorFlow model using SageMaker with distributed training. They need to implement data parallelism across multiple GPUs. Which SageMaker feature should they use to distribute the training?

Accepted Answer

SageMaker Distributed Data Parallelism. SageMaker's distributed data parallelism library handles splitting data across GPUs and synchronizing gradients, optimized for TensorFlow and PyTorch.

Answer

SageMaker Automatic Model Tuning

Answer

SageMaker Debugger

Answer

SageMaker Model Parallelism