MLA-C01 ML Model Development Practice Test 7 — 15 Questions

Question 1

A data scientist is using SageMaker to train a model and wants to reduce training costs without sacrificing performance. Which TWO actions should the scientist take? (Select TWO.)

Accepted Answer

Use SageMaker managed spot training. Using spot instances can reduce costs up to 90%. SageMaker managed spot training handles interruptions automatically. Using distributed training across multiple smaller instances can be cost-effective compared to a single large instance. Using Provisioned Concurrency is for inference, not training. Debugger hooks do not reduce cost.

Answer

Use a larger instance type to finish faster

Answer

Enable SageMaker Debugger hooks to monitor training

Answer

Enable SageMaker Model Monitor for the training job

Question 2

An ML engineer is fine-tuning a foundation model using RLHF on SageMaker. Which THREE components are essential for this workflow? (Select THREE.)

Accepted Answer

A reward model trained on the preference data. RLHF requires a preference dataset for human feedback, a reward model trained on that data, and the PPO algorithm to update the foundation model. The PEFT technique (like LoRA) is often used to make fine-tuning efficient, but it is not strictly essential for RLHF; however, it is commonly used. The base foundation model is required. A validation dataset is needed but not specific to RLHF.

Answer

A large validation dataset for final evaluation

Answer

A PEFT technique like LoRA

Question 3

A team wants to evaluate a binary classification model for credit risk. They need to understand the trade-off between false positives and false negatives. Which TWO metrics should they use? (Select TWO.)

Accepted Answer

Recall. Precision and recall are complementary; precision measures false positives, recall measures false negatives. AUC-ROC summarizes the trade-off across thresholds. RMSE is for regression. NDCG is for ranking.

Answer

NDCG

Answer

AUC-ROC

Answer

RMSE

Question 4

A data scientist needs to train a binary classification model on a large tabular dataset stored in Amazon S3. The team wants to minimize training time and cost while using a built-in SageMaker algorithm. Which algorithm should they use?

Accepted Answer

Linear Learner. Linear Learner is a built-in SageMaker algorithm designed for binary classification and regression, and it scales efficiently on large datasets. XGBoost is better for structured data with non-linear relationships, DeepAR is for time series, and BlazingText is for text.

Answer

BlazingText

Answer

DeepAR

Answer

XGBoost

Question 5

A team is training a large deep learning model on SageMaker using a single ml.p3.16xlarge instance. Training is taking too long. They want to reduce time by distributing across multiple GPUs but are constrained by model size that does not fit in a single GPU memory. Which distributed training strategy should they use?

Accepted Answer

Model parallelism using SageMaker distributed model parallelism. Model parallelism splits the model across multiple GPUs, which is needed when the model does not fit in a single GPU. Data parallelism replicates the model on each GPU and splits data, which requires the model to fit in each GPU's memory.

Answer

Data parallelism using SageMaker distributed data parallelism

Answer

Switch to a smaller instance type and use horizontal scaling

Answer

Use multiple training jobs with hyperparameter tuning

Question 6

A company is using SageMaker Automatic Model Tuning to optimize a regression model. They want to minimize the root mean squared error (RMSE). The tuner has completed 20 jobs, and the RMSE has plateaued. Which action should the data scientist take to potentially improve the results?

Accepted Answer

Decrease the range of hyperparameters to focus on promising areas. Reducing the search space can help the tuner focus on more promising regions. Increasing parallelism or max jobs may explore the same plateau, while switching to a different algorithm altogether might not be necessary.

Answer

Increase the maximum number of training jobs

Answer

Increase the number of parallel training jobs

Answer

Switch the objective metric to mean absolute error (MAE)

Question 7

A machine learning engineer is using Amazon SageMaker Debugger to monitor a training job for a deep neural network. They receive a rule alert indicating 'exploding gradients'. Which action should they take to address this issue?

Accepted Answer

Reduce the learning rate. Exploding gradients occur when gradients become too large, causing instability. Reducing the learning rate mitigates this. Increasing batch size can also help by smoothing gradients, but reducing learning rate is a direct solution.

Answer

Use a smaller batch size

Answer

Increase the number of layers to absorb gradients

Answer

Increase the learning rate

Question 8

A data scientist is using SageMaker Autopilot to automatically build a binary classification model on a balanced dataset. They want to understand the relationship between the input features and the model predictions. Which feature in SageMaker Autopilot should they use?

Accepted Answer

Explainability reports. SageMaker Autopilot generates explainability reports, including feature importance and model insights, via the 'Explainability' feature. This provides the relationship between features and predictions.

Answer

Data visualizations

Answer

Model tuning results

Answer

Model candidate definitions

Question 9

A team wants to fine-tune a pre-trained Hugging Face transformer model for text classification using SageMaker. They have a custom training script. Which SageMaker estimator should they use?

Accepted Answer

SageMaker Hugging Face estimator. The Hugging Face estimator is the recommended way to run Hugging Face models on SageMaker, as it automatically handles the environment and dependencies.

Answer

SageMaker generic estimator with a custom container

Answer

SageMaker PyTorch estimator

Answer

SageMaker TensorFlow estimator

Question 10

A financial services company trains multiple models on SageMaker and needs to track hyperparameters, metrics, and artifacts for each experiment. Which SageMaker feature should they use to organize and compare experiments?

Accepted Answer

SageMaker Experiments. SageMaker Experiments provides experiment management, allowing users to track parameters, metrics, and artifacts, and compare runs. SageMaker Studio offers an interface but the core feature is Experiments.

Answer

SageMaker Model Registry

Answer

SageMaker Pipelines

Answer

SageMaker Debugger

Question 11

A company is fine-tuning a large language model using LoRA on SageMaker. They want to reduce GPU memory usage during training. Which configuration change would help?

Accepted Answer

Use QLoRA (quantized LoRA) with 4-bit quantization. LoRA reduces trainable parameters, and when combined with QLoRA (quantized LoRA), it further reduces memory by quantizing the base model to 4-bit or 8-bit. Increasing batch size or sequence length typically increases memory usage. Gradient accumulation also increases memory as it requires storing gradients for multiple steps. QLoRA is specifically designed for memory reduction.

Answer

Enable gradient accumulation

Answer

Increase the sequence length

Answer

Increase the batch size

Question 12

A data scientist is using SageMaker to train a model and wants to reduce training costs without sacrificing performance. Which TWO actions should the scientist take? (Select TWO.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Mean Absolute Error (MAE). MAE (Mean Absolute Error) gives the average absolute prediction error, which is easy to interpret in the same units as the target. RMSE gives a similar but squared metric, and R² indicates variance explained, but MAE directly answers the average error magnitude.

Answer

AUC

Answer

Confusion matrix

Answer

F1 score

Question 13

A company wants to detect anomalies in login events from a large user base, focusing on unusual patterns that may indicate compromised accounts. Which SageMaker built-in algorithm is most suitable for this task?

Accepted Answer

IP Insights. IP Insights is designed for anomaly detection in IP address usage, learning typical login patterns and flagging unusual ones. The other algorithms are not specialized for this use case.

Answer

K-Means

Answer

DeepAR

Answer

Factorisation Machines

Question 14

A team is using SageMaker to train a distributed model with data parallelism. They notice that the training loss is not decreasing as expected and suspect a bug in the data loading pipeline. Which SageMaker Debugger feature can help them inspect the data distributions during training?

Accepted Answer

Custom rules to monitor input tensors. SageMaker Debugger can capture tensors (including inputs and outputs) during training. By saving input tensors, the team can inspect data distributions. Rules fire on issues like overfitting or dead relu, but tensor captures allow direct data inspection. SaveConfig defines which tensors to save. Using a different instance type does not help debug data issues.

Answer

SageMaker Model Monitor

Answer

Built-in rules such as overfit detection

Answer

SageMaker Processing jobs

Question 15

A data scientist needs to run a hyperparameter tuning job for a PyTorch model using SageMaker. They want to use Hyperband for efficient resource allocation. Which tuning strategy should they select in the HyperparameterTuner?

Accepted Answer

Hyperband. SageMaker Automatic Model Tuner supports Bayesian, Random, and Hyperband strategies. Hyperband is an early stopping-based method that allocates resources adaptively. The 'Hyperband' strategy should be selected explicitly.

Answer

Bayesian optimization

Answer

Random search

Answer

Grid search