MLA-C01 ML Model Development Practice Test 6 — 15 Questions

Question 1

A data science team is using Amazon SageMaker Pipelines to orchestrate a multi-step workflow that includes data preprocessing, training, and model evaluation. They want to reuse the preprocessed data across multiple pipeline executions without re-running the preprocessing step if the source data hasn't changed. What should they configure?

Accepted Answer

Use SageMaker Processing steps with caching. SageMaker Pipelines supports step caching, which allows reusing the output of a step if its inputs and parameters are unchanged. SageMaker Feature Store is for feature storage and serving, not for pipeline step reuse. Checkpointing is for training resumption, not step caching. Data Wrangler preprocesses but caching is a pipeline feature.

Answer

Use SageMaker Training steps with checkpointing

Answer

Use SageMaker Feature Store to store the preprocessed features

Answer

Use SageMaker Data Wrangler for the preprocessing

Question 2

A data scientist is preparing text data for a sentiment analysis model using Amazon SageMaker. Which two data preprocessing techniques are commonly used when working with text data for natural language processing? (Choose two.)

Accepted Answer

Tokenization. Stop word removal and tokenization are standard text preprocessing steps. One-hot encoding of all words leads to high dimensionality and is rarely used directly. Image resizing is for images, and PCA is for numerical dimensionality reduction.

Answer

One-hot encoding of all words

Answer

Image resizing

Answer

Principal component analysis (PCA)

Question 3

A machine learning engineer is evaluating a binary classification model for detecting fraudulent transactions. The dataset is highly imbalanced, and the cost of false negatives (missing a fraud) is very high. Which two evaluation metrics should the engineer consider? (Choose two.)

Accepted Answer

F1-score. Recall captures the proportion of actual frauds correctly identified, directly addressing false negatives. F1-score balances precision and recall, providing a single score. Accuracy is misleading on imbalanced data, precision focuses on false positives, and mean absolute error is for regression.

Answer

Accuracy

Answer

Precision

Answer

Mean absolute error

Question 4

A team is training a deep learning model on Amazon SageMaker using a custom Docker container. Which three practices should they follow to optimize training performance? (Choose three.)

Accepted Answer

Store training data in Amazon S3 in a shuffled and compressed format. Storing training data in Amazon S3 in a shuffled and compressed format (Option A) optimizes training performance because shuffling prevents biased gradient updates during stochastic gradient descent, while compression reduces I/O overhead and network transfer time. SageMaker's Pipe mode can then stream this compressed data directly to the training algorithm without intermediate disk writes, further accelerating throughput.

Answer

Use the largest instance type available

Answer

Increase the number of layers in the model to improve accuracy

Question 5

What is the most likely cause of this error?

Accepted Answer

The account has not requested a service limit increase for the specified instance type.. The error indicates that the service limit for the specified instance type is set to 0, meaning the account has not been granted a limit increase for that instance. The instance might be available in the region, but the limit is zero. IAM permissions would give an access denied error, not ResourceLimitExceeded. Spot unavailability generates a different error.

Answer

The training job is configured to use Managed Spot Training and the spot market is unavailable.

Answer

The IAM role used for the training job does not have sufficient permissions to launch the instance.

Answer

The instance type specified is not available in the current AWS Region.

Question 6

What will the debugger do with this configuration?

Accepted Answer

It will capture gradients every 500 steps and trigger a rule if loss does not decrease for 10 steps with a threshold of 0.001.. The 'save_interval' in the collection captures gradients every 500 steps. The rule 'LossNotDecreasing' checks if the loss does not decrease for 'patience' consecutive steps (10) within a tolerance of 'threshold' (0.001). Option B incorrectly interprets the timing; option C swaps values; option D incorrectly states rules run despite the misspelling? Actually rule name is valid.

Answer

It will only capture gradients and not run any rules because the rule name is misspelled.

Answer

It will capture gradients every 10 steps and trigger a rule if loss does not decrease for 500 epochs.

Answer

It will capture gradients every 500 steps and trigger a rule if loss does not decrease for 500 iterations with a patience of 10.

Question 7

The training job completes successfully but the model performance is poor. What is a likely cause?

Accepted Answer

The max_depth hyperparameter is too high, leading to overfitting.. A max_depth of 10 is high for many datasets, often leading to overfitting. Overfitting results in poor generalization. The other options are less likely: num_round=50 is moderate, instance type is not directly related to model performance, and data shuffling is not specified but the primary issue is hyperparameter choice.

Answer

The training data is not shuffled.

Answer

The instance type is too small for the dataset.

Answer

The number of rounds (num_round) is too high.

Question 8

A data scientist is training a linear regression model on a dataset with 10 features. After training, the model shows high training accuracy but poor test accuracy. Which of the following is the most likely cause?

Accepted Answer

Overfitting. High training accuracy and poor test accuracy indicates overfitting. Underfitting would show poor training accuracy. Data leakage could cause high accuracy but not necessarily overfitting. Feature scaling is a preprocessing step and not directly a cause of this behavior.

Answer

Data leakage

Answer

Feature scaling

Answer

Underfitting

Question 9

A company uses Amazon SageMaker to train a custom XGBoost model. The training job runs on a single ml.m5.large instance and takes 2 hours. To reduce training time without changing the algorithm, what should the data scientist do?

Accepted Answer

Use a larger instance type. Using a larger instance type (e.g., ml.m5.xlarge) provides more compute capacity, directly reducing training time. Automatic model tuning adds overhead and does not reduce time. Built-in XGBoost runs faster but changes the algorithm. Increasing epochs would increase training time.

Answer

Increase the number of epochs

Answer

Use SageMaker's built-in XGBoost algorithm

Answer

Enable automatic model tuning

Question 10

A machine learning team is developing a deep learning model for image classification. They observe that the training loss decreases rapidly but the validation loss starts increasing after a few epochs. Which strategy should they implement to address this issue?

Accepted Answer

Apply dropout regularization. Increasing validation loss indicates overfitting. Dropout regularization randomly drops neurons during training, which reduces overfitting. Increasing learning rate would make training unstable. Adding more layers increases capacity and likely worsens overfitting. Increasing batch size can have a regularizing effect but is not as direct as dropout.

Answer

Increase the batch size

Answer

Add more convolutional layers

Answer

Increase the learning rate

Question 11

A data scientist wants to evaluate the performance of a binary classification model. The dataset is highly imbalanced with only 5% positive class. Which metric should be used to evaluate the model?

Accepted Answer

F1-score. F1-score balances precision and recall, making it suitable for imbalanced datasets. Accuracy can be misleading (e.g., 95% if always predicting negative). Mean Squared Error and R-squared are for regression.

Answer

Accuracy

Answer

Mean Squared Error

Answer

R-squared

Question 12

During model training on Amazon SageMaker, the training job fails with a 'ResourceLimitExceeded' error. What is the most likely cause?

Accepted Answer

The account's instance limit for the chosen instance type has been reached. The ResourceLimitExceeded error typically indicates that the AWS account has reached its service limit for the specified instance type. A syntax error would cause a different error (e.g., ClientError). A dataset too large might cause an out-of-memory error but not this specific error. Learning rate does not cause resource limits.

Answer

The algorithm's learning rate is too high

Answer

The dataset is too large for the instance

Answer

The training script has a syntax error

Question 13

A data scientist is using Amazon SageMaker Debugger to monitor training metrics. They want to stop training automatically if the model is overfitting. Which action should they take?

Accepted Answer

Configure a custom rule that triggers a STOP training action when validation loss stops decreasing. SageMaker Debugger allows custom rules with actions like STOP training. A built-in rule for overfitting may not exist, so a custom rule is needed. The rule should check if validation loss stops decreasing (plateau) or starts increasing, and trigger STOP. Other options monitor different issues.

Answer

Define a Debugger rule that monitors the loss plateau

Answer

Create a SageMaker Training Compiler

Answer

Use a built-in rule that checks for vanishing gradients

Question 14

Which of the following is a recommended practice for preparing training data in Amazon SageMaker?

Accepted Answer

Convert data to RecordIO format for built-in algorithms. RecordIO format is recommended for built-in algorithms as it improves I/O performance. Storing data in Amazon DynamoDB is not suitable for training datasets. Public read access on S3 is insecure. While gzip compression is common, RecordIO is a specific best practice for SageMaker.

Answer

Store training data in Amazon DynamoDB

Answer

Compress data using gzip to reduce transfer time

Answer

Use Amazon S3 with public read access

Question 15

A data scientist is training a logistic regression model and wants to use L1 regularization to create a sparse model. Which parameter should be adjusted?

Accepted Answer

penalty. In scikit-learn's LogisticRegression, the 'penalty' parameter can be set to 'l1' to use L1 regularization. 'C' is the inverse of regularization strength, but without setting penalty='l1', it won't be L1. 'alpha' and 'lambda' are parameters in other libraries like scikit-learn's linear models (alpha) or XGBoost (lambda), but not for logistic regression default.

Answer

alpha

Answer

lambda

Answer

C (inverse of regularization strength)