MLA-C01 ML Model Development Practice Test 3 — 15 Questions

Question 1

An ML team is using SageMaker Automatic Model Tuning to optimize hyperparameters for a neural network. They want to prioritize exploration of the hyperparameter space early in the tuning process. Which strategy should they choose?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Bayesian optimization. Bayesian optimization builds a probabilistic surrogate model of the objective function and uses an acquisition function to choose the next hyperparameter set, which balances exploration and exploitation. This makes it more sample-efficient than grid or random search and is the strategy SageMaker Automatic Model Tuning uses by default to prioritize promising regions early. It directly satisfies the goal of prioritizing exploration of the hyperparameter space.

Answer

Grid search

Answer

Random search

Answer

Hyperband

Question 2

Which SageMaker feature automatically generates model cards, feature importance, and bias reports without requiring manual coding?

Accepted Answer

SageMaker Autopilot. SageMaker Autopilot is an automated machine learning (AutoML) service that automatically explores data, selects algorithms, trains and tunes models, and generates explainability reports including model cards, feature importance, and bias detection—all without writing code. It integrates SageMaker Clarify under the hood to produce these artifacts as part of the AutoML process. The key differentiator is that Autopilot provides these outputs automatically as part of its end-to-end pipeline, whereas other services require manual configuration or coding.

Answer

SageMaker Experiments

Answer

SageMaker Clarify

Answer

SageMaker Model Monitor

Question 3

A data scientist is using SageMaker to train a model and wants to reduce training costs without sacrificing performance. Which TWO actions should the scientist take? (Select TWO.)

Accepted Answer

Use SageMaker managed spot training. Option B is correct because SageMaker managed spot training lets the training job run on spare EC2 capacity at up to a 90% discount versus on-demand, and SageMaker automatically manages the checkpointing and interruption handling so the job resumes without losing progress, directly reducing cost without changing model quality. Option E is correct because distributed training splits the workload across multiple smaller instances, so the same training completes in comparable time using cheaper aggregate compute rather than one expensive large instance, lowering cost while preserving performance. Option A is not appropriate because a larger instance type raises the hourly cost and does not by itself reduce total training expense. Option C is not appropriate because SageMaker Debugger hooks add monitoring and profiling overhead and do not lower training cost. Option D is not appropriate because SageMaker Model Monitor is for detecting drift and quality issues on deployed endpoints, not for reducing training job costs.

Answer

Use a larger instance type to finish faster

Answer

Enable SageMaker Debugger hooks to monitor training

Answer

Enable SageMaker Model Monitor for the training job

Question 4

A data scientist needs to train a binary classification model on a large tabular dataset stored in Amazon S3. The team wants to minimize training time and cost while using a built-in SageMaker algorithm. Which algorithm should they use?

Accepted Answer

Linear Learner. Linear Learner is a built-in SageMaker algorithm designed for binary classification on tabular data. It is highly efficient for large datasets because it uses stochastic gradient descent and supports distributed training across multiple instances. Unlike XGBoost, which is also a strong choice for tabular data, Linear Learner is optimized for speed and cost when the relationship between features and target is approximately linear, and it scales well with data size. Thus, it minimizes training time and cost while meeting the requirement of a built-in algorithm for binary classification.

Answer

BlazingText

Answer

DeepAR

Answer

XGBoost

Question 5

A team is using SageMaker to train a distributed model with data parallelism. They notice that the training loss is not decreasing as expected and suspect a bug in the data loading pipeline. Which SageMaker Debugger feature can help them inspect the data distributions during training?

Accepted Answer

Custom rules to monitor input tensors. SageMaker Debugger can capture tensors (including inputs and outputs) during training. By saving input tensors, the team can inspect data distributions. Rules fire on issues like overfitting or dead relu, but tensor captures allow direct data inspection. SaveConfig defines which tensors to save. Using a different instance type does not help debug data issues.

Answer

SageMaker Model Monitor

Answer

Built-in rules such as overfit detection

Answer

SageMaker Processing jobs

Question 6

A data scientist is evaluating a binary classification model for loan default prediction. Which THREE metrics should they consider to thoroughly assess model performance, especially for imbalanced classes?

Accepted Answer

Recall. Recall (B) is correct because it measures the proportion of actual defaulters that the model successfully identifies (TP/(TP+FN)), which is critical when missing a true default is costly and the positive class is rare. F1 score (D) is correct because it is the harmonic mean of precision and recall, giving a balanced single metric that penalizes poor performance on either, making it well suited to imbalanced binary classification. AUC (E), specifically the area under the ROC curve, is correct because it evaluates the model's ability to rank positives above negatives across all classification thresholds, independent of class distribution. R² (A) is not appropriate because it measures explained variance in regression, not classification quality. RMSE (C) is also a regression error metric measuring average squared prediction error, so it does not assess classification performance on imbalanced classes.

Answer

R²

Answer

RMSE

Question 7

During a SageMaker training job, the loss stops decreasing and the validation accuracy plateaus early. SageMaker Debugger rules are enabled. Which rule is MOST likely to identify this issue?

Accepted Answer

Overfit rule. The Overfit rule in SageMaker Debugger monitors the gap between training loss and validation loss; when training loss continues to decrease while validation loss stops improving or increases, the rule flags overfitting. A plateau in validation accuracy with stalled training loss decrease is the textbook overfitting signature.

Answer

Weight distribution rule

Answer

Exploding gradients rule

Answer

Dead relu rule

Question 8

Which SageMaker built-in algorithm is designed for time series forecasting?

Accepted Answer

DeepAR. DeepAR is a SageMaker built-in algorithm specifically designed for time series forecasting. It uses recurrent neural networks (RNNs) to model temporal dependencies and can handle multiple related time series, making it ideal for forecasting tasks.

Answer

BlazingText

Answer

IP Insights

Answer

XGBoost

Question 9

A team is fine-tuning a foundation model using LoRA. They want to reduce memory usage during training. Which technique should they combine LoRA with to further reduce memory?

Accepted Answer

QLoRA. QLoRA (Quantized LoRA) extends LoRA by quantizing the frozen base model weights to 4-bit precision (typically NF4) while keeping the LoRA adapters in higher precision during training. This dramatically reduces GPU memory because the base model — which is the largest memory consumer — is stored in 4-bit instead of 16-bit or 32-bit, while still allowing effective fine-tuning of the low-rank adapters. Combining LoRA with QLoRA is the standard technique for reducing memory during fine-tuning of large foundation models.

Answer

Instruction tuning

Answer

Pruning

Answer

RLHF

Question 10

A data scientist is using SageMaker built-in Image Classification algorithm on a dataset with 1000 classes. The training is very slow. They want to speed it up without sacrificing accuracy. Which instance type and training configuration is MOST appropriate?

Accepted Answer

Use ml.p3.16xlarge instances with data parallelism. The SageMaker Image Classification algorithm is a GPU-accelerated deep learning algorithm, so it requires GPU instances to train efficiently. ml.p3.16xlarge provides 8 NVIDIA V100 GPUs, and data parallelism distributes the training data across multiple GPUs to accelerate training without changing the model architecture or reducing accuracy. This combination directly addresses the slow training on a 1000-class dataset by leveraging GPU compute and horizontal scaling.

Answer

Use ml.trn1.32xlarge instances with data parallelism

Answer

Use ml.m5.24xlarge instances with data parallelism

Answer

Use ml.c5.18xlarge instances with model parallelism

Question 11

A machine learning engineer is deploying a custom PyTorch model using SageMaker script mode. The training script requires specific dependencies not included in the default PyTorch container. Which TWO actions can the engineer take to ensure the dependencies are available? (Select TWO.)

Accepted Answer

Build a custom container that extends the SageMaker PyTorch container and push it to Amazon ECR. Option A is correct because SageMaker script mode allows you to bring your own container: you can extend the official PyTorch container (e.g., FROM 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:...), install the extra packages in the Dockerfile, push the image to Amazon ECR, and pass its URI to the estimator's image_uri so the dependencies are baked into the runtime. Option B is correct because when using the prebuilt PyTorch container in script mode, SageMaker automatically installs packages listed in a requirements.txt placed in the source directory (source_dir) before running the training script, which is the standard way to add extra Python dependencies. Option C is wrong because lifecycle configurations apply to notebook instances, not to SageMaker training jobs, so they cannot install packages inside the training container. Option D is wrong because specifying a custom Docker image is essentially the same mechanism as Option A but is not a valid standalone estimator parameter in the way described; the supported approach is to set image_uri on the estimator, and the option as worded does not reflect the correct script-mode action. Option E is wrong because source_dir is a single directory of source code, not a place to attach a separate container, and you cannot add dependencies by passing them as a container through that argument.

Answer

Use a lifecycle configuration to install dependencies

Answer

Specify a custom Docker image in the PyTorch estimator

Answer

Add the dependencies to the estimator's source_dir argument as a separate container

Question 12

A company is using SageMaker built-in XGBoost algorithm for a multiclass classification problem. They want to evaluate the model's performance. Which TWO metrics are appropriate for multiclass classification? (Select TWO.)

Accepted Answer

Precision (macro or weighted). Precision (macro or weighted) (B) is appropriate because it measures, for each class, the fraction of predicted positives that are truly correct, and macro/weighted averaging extends this to multiclass settings by aggregating per-class precision. Recall (macro or weighted) (C) is likewise appropriate because it measures the fraction of actual instances of each class that the model correctly retrieves, with macro/weighted averaging summarizing performance across all classes. Both are standard classification metrics reported by SageMaker's XGBoost evaluation for multiclass problems. RMSE (A) is a regression metric measuring squared error of continuous predictions, not class labels. NDCG (D) is a ranking metric for graded relevance in search/recommendation, not classification. AUC (E) is defined for binary classification (or one-vs-rest extensions), and while it can be adapted, it is not the standard multiclass classification metric expected here.

Answer

RMSE

Answer

NDCG (Normalized Discounted Cumulative Gain)

Answer

AUC (Area Under the ROC Curve)

Question 13

A data scientist is training a large language model using SageMaker and wants to reduce training costs. The training job is expected to run for several days. Which TWO actions should the data scientist take to minimize costs? (Choose TWO.)

Accepted Answer

Enable managed spot training. Option A (Enable managed spot training) is correct because SageMaker Managed Spot Training lets you run training jobs on spare EC2 capacity at up to a 90% discount compared to on-demand pricing, which directly reduces cost for a multi-day job; you just need to configure checkpointing to S3 so the job can resume after interruptions. Option C (Select ml.g5.xlarge instead of ml.p3.2xlarge) is correct because the ml.g5.xlarge is a newer-generation, more cost-effective GPU instance that is generally cheaper per hour than the older ml.p3.2xlarge while still providing a comparable NVIDIA GPU for many LLM training workloads, lowering the hourly rate. Option B is wrong because choosing the most powerful (and most expensive) GPU instance increases the hourly cost and is not guaranteed to reduce total cost even if it finishes faster. Option D is wrong because adding more instances increases the total compute cost, and distributed training overhead can offset any time savings. Option E is wrong because disabling checkpointing is risky and does not meaningfully reduce cost; in fact, checkpointing is required to benefit from Spot training and to avoid losing progress.

Answer

Use the most powerful GPU instance available to finish faster

Answer

Increase the number of instances to reduce time

Answer

Disable checkpointing to save storage costs

Question 14

A company uses SageMaker Autopilot to build a regression model predicting house prices. After the experiment completes, the company wants to understand why the model makes certain predictions. Which TWO SageMaker features can provide this explainability? (Choose TWO.)

Accepted Answer

SageMaker Clarify. SageMaker Clarify (A) is correct because it provides feature attribution explanations using SHAP values, showing how each input feature contributes to a model's prediction, which directly answers why the regression model predicts a given house price. SageMaker Autopilot explainability report (B) is correct because Autopilot automatically generates an explainability report for the best model, including feature importance and SHAP-based explanations, giving insight into the model's predictions without additional setup. SageMaker Model Monitor (C) is not correct because it detects data drift, model quality degradation, and bias in production, not the reasoning behind individual predictions. SageMaker Debugger (D) is not correct because it focuses on debugging training jobs by capturing tensors, metrics, and resource utilization, not on explaining predictions. SageMaker Experiments (E) is not correct because it tracks and compares training runs, parameters, and metrics, but does not provide prediction explainability.

Answer

SageMaker Model Monitor

Answer

SageMaker Debugger

Answer

SageMaker Experiments

Question 15

A team is using SageMaker Automatic Model Tuning to optimize hyperparameters for an XGBoost model. They want to find the best configuration as quickly as possible, with a maximum of 50 training jobs. Which TWO strategies should they choose? (Choose TWO.)

Accepted Answer

Use Hyperband with early stopping. Option B (Hyperband with early stopping) is correct because Hyperband is a multi-fidelity, bandit-based strategy that aggressively terminates poorly performing training jobs early, reallocating resources to promising configurations, which makes it the fastest way to explore many configurations within a 50-job budget. Option E (Bayesian optimization) is correct because it is SageMaker Automatic Model Tuning's default strategy that builds a probabilistic model of the objective function to intelligently select the next hyperparameter set, converging on the best configuration in fewer jobs than uninformed search. Option C (random search) is not marked correct because, while it is a valid SageMaker strategy, it does not adapt based on prior results and is generally slower to converge than Bayesian optimization or Hyperband. Option D (grid search) is not marked correct because it exhaustively evaluates a fixed Cartesian product of values, which scales poorly and is inefficient for a 50-job limit. Option A is not marked correct because it is not a tuning strategy at all—simply reusing the same objective metric with different strategies is not a defined SageMaker approach for accelerating tuning.

Answer

Use the same objective metric but with different strategies

Answer

Use random search

Answer

Use grid search