MLS-C01 Modeling Practice Test 19 — 15 Questions

Question 1

A data scientist is using Amazon SageMaker Automatic Model Tuning to optimize a regression model. The objective metric is validation RMSE, and the tuner is configured with the Bayesian optimization strategy. After several hours, the tuner has completed many training jobs but the best objective value has barely improved, and the scientist suspects the search is stuck in a poor region of the hyperparameter space. Which action is MOST likely to improve the tuning results?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Widen the hyperparameter ranges and adjust the objective metric to use the average of the last epoch.. When Bayesian optimization stalls, the usual causes are an overly narrow search space or a noisy objective that misleads the surrogate model. Widening the ranges lets the tuner explore promising regions it previously could not reach, and averaging the metric over the last epoch reduces noise so the surrogate model receives more reliable feedback about each configuration's true performance.

Answer

Switch the tuning strategy to Random and increase the number of parallel training jobs.

Answer

Reduce max_jobs and max_parallel_jobs to force the tuner to evaluate fewer configurations.

Answer

Change the objective metric to training loss instead of validation RMSE.

Question 2

A machine learning engineer is tuning a gradient boosting model on Amazon SageMaker. The training dataset is small (1,000 rows) and contains 50 features, many of which are correlated. The engineer notices that the model performs well on the training set but poorly on the validation set. Which SageMaker built-in algorithm and hyperparameter configuration is MOST appropriate to reduce overfitting while maintaining interpretability?

Accepted Answer

Use the XGBoost algorithm with a lower value for max_depth (e.g., 3) and a higher value for lambda (e.g., 10).. For small datasets with many correlated features, overfitting is a common issue. XGBoost with shallow trees (low max_depth) and strong L2 regularization (high lambda) controls model complexity effectively. This combination reduces variance without sacrificing the algorithm's ability to model nonlinear interactions, resulting in better generalization and interpretability through feature importance scores.

Answer

Use the Linear Learner algorithm with L1 regularization and tune the regularization parameter.

Answer

Use the XGBoost algorithm with a high value for max_depth (e.g., 10) and a low value for lambda (e.g., 0).

Answer

Use the K-Nearest Neighbors algorithm with a small value for k (e.g., 1) and Euclidean distance.

Question 3

A machine learning engineer is using Amazon SageMaker to train an image classification model with the built-in Image Classification algorithm. The engineer wants to use transfer learning with a pre-trained model. Which hyperparameter must be set to use a pre-trained model from a different dataset?

Accepted Answer

`use_pretrained_model`. The correct answer is `use_pretrained_model`. In SageMaker's Image Classification algorithm, setting this hyperparameter to 1 enables transfer learning by initializing the network with weights pre-trained on ImageNet. This is particularly useful when the training dataset is small, as it leverages features learned from a large dataset. Other hyperparameters like num_layers, epochs, and learning_rate affect training but do not control the use of pre-trained weights. The engineer must explicitly set `use_pretrained_model` to benefit from transfer learning.

Answer

`num_layers`

Answer

`epochs`

Answer

`learning_rate`

Question 4

A machine learning engineer is using Amazon SageMaker to train a deep learning model for image classification. The training dataset is stored in Amazon S3 and consists of 500,000 images, totaling 1 TB. The engineer notices that each epoch takes a very long time and wants to optimize the training speed. The model uses the built-in Image Classification algorithm with a ResNet-50 backbone. Which of the following changes is MOST likely to reduce the training time without significantly affecting model accuracy?

Accepted Answer

Increase the number of GPU instances in the training cluster and enable distributed training.. Distributed training across multiple GPU instances is the most effective way to reduce training time for large-scale deep learning without altering the model architecture or data resolution. SageMaker's built-in Image Classification algorithm supports data parallelism, which splits the training data across instances and aggregates gradients. This approach maintains accuracy when hyperparameters are adjusted, and it directly addresses the computational bottleneck.

Answer

Use SageMaker Spot Instances for the training job to reduce cost, which indirectly speeds up training.

Answer

Convert the images to the TFRecord format and use Pipe mode for data input.

Answer

Reduce the image resolution from 224x224 to 128x128.

Question 5

A machine learning engineer is using Amazon SageMaker to train a model with the built-in XGBoost algorithm. They want to reduce overfitting and improve model generalization. Which two hyperparameter adjustments should they consider? (Choose two.)

Accepted Answer

Decrease the value of eta (learning rate).. To reduce overfitting in XGBoost, increasing L2 regularization (lambda) and decreasing the learning rate (eta) are effective. Higher lambda penalizes complex models, while lower eta requires more trees but often yields better generalization. Other options like increasing max_depth or subsample would increase model complexity or reduce regularization, worsening overfitting.

Answer

Increase the value of max_depth.

Answer

Set the objective to reg:squarederror.

Answer

Increase the value of subsample.

Question 6

A machine learning engineer is training a linear regression model on a dataset where the target variable has a highly skewed distribution. The team wants to reduce the impact of extreme outliers on the model's coefficients without removing any data points. Which approach should the engineer use?

Accepted Answer

Apply a logarithmic transformation to the target variable before training.. Transforming a skewed target with a logarithm reduces the relative weight of extreme values, making the regression less sensitive to outliers while retaining all data. This is a common preprocessing step in Amazon SageMaker workflows. Other options either do not address the skew or alter the model in ways that may harm performance without solving the core issue.

Answer

Standardize the target variable by subtracting the mean and dividing by the standard deviation.

Answer

Use a Huber loss function instead of mean squared error.

Answer

Increase the L2 regularization strength (lambda) to a very high value.

Question 7

A data scientist is using Amazon SageMaker to train a model on a dataset that contains a large number of categorical features with high cardinality. The data scientist wants to reduce the dimensionality of these features before training a linear model. Which SageMaker built-in algorithm is specifically designed for this purpose?

Accepted Answer

Use the SageMaker Principal Component Analysis (PCA) algorithm.. The SageMaker PCA algorithm is specifically designed for dimensionality reduction by projecting data onto principal components. It is commonly used to reduce the dimensionality of high-dimensional data, including one-hot encoded categorical features, before training a linear model. The other algorithms serve different purposes: Object2Vec is for embedding learning, Factorization Machines are for supervised prediction with interactions, and K-Means is for clustering.

Answer

Use the SageMaker Object2Vec algorithm.

Answer

Use the SageMaker Factorization Machines algorithm.

Answer

Use the SageMaker K-Means algorithm.

Question 8

A machine learning team is using Amazon SageMaker to train a model with the built-in Linear Learner algorithm on a dataset with 100 features and 1 million samples. The team wants to perform binary classification and needs to handle missing values and categorical features. They also want to automatically tune hyperparameters to maximize the area under the ROC curve (AUC). Which combination of steps should they take to meet these requirements?

Accepted Answer

Use the SageMaker Linear Learner algorithm with 'predictor_type' set to 'binary_classifier', set 'binary_classifier_model_selection_criteria' to 'auc', specify categorical features via the 'categorical_features' hyperparameter, and enable automatic model tuning with AUC as the objective metric.. Linear Learner in SageMaker supports binary classification and can directly optimize for AUC via the 'binary_classifier_model_selection_criteria' hyperparameter. By specifying categorical features, the algorithm treats missing values as a separate category, eliminating the need for imputation. Enabling automatic model tuning with AUC as the objective metric will search for optimal hyperparameters. This approach efficiently addresses missing values, categorical features, and hyperparameter tuning.

Answer

Use the SageMaker Linear Learner algorithm with 'predictor_type' set to 'binary_classifier', enable 'use_bias', and perform one-hot encoding for categorical features and imputation for missing values in a SageMaker Processing job.

Answer

Use the SageMaker Linear Learner algorithm with the 'binary_classifier_model_selection_criteria' set to 'auc', enable 'use_bias' and handle missing values by replacing them with zeros in a preprocessing step.

Answer

Use the SageMaker XGBoost algorithm instead because it handles missing values and categorical features natively and supports automatic model tuning for AUC.

Question 9

A machine learning team is using Amazon SageMaker to train a model for predicting customer churn. The training dataset is stored in Amazon S3 and contains 100,000 records with 20 features. The team wants to use the SageMaker Debugger to monitor training and detect issues such as vanishing gradients. They configure a training job with the XGBoost algorithm. Which of the following is TRUE regarding SageMaker Debugger's capabilities for this job?

Accepted Answer

SageMaker Debugger can collect metrics like training loss and validation loss, and apply built-in rules to detect overfitting for XGBoost.. SageMaker Debugger supports metric-based monitoring for algorithms like XGBoost. It can collect training and validation loss metrics and apply built-in rules such as Overfit to detect when the model is overfitting. This is valuable for the churn prediction task, enabling the team to take corrective actions like early stopping or regularization.

Answer

SageMaker Debugger can automatically detect vanishing gradients and stop the training job.

Answer

SageMaker Debugger can capture tensors from XGBoost training and apply built-in rules to detect overfitting.

Answer

SageMaker Debugger can monitor the training job's resource utilization and provide recommendations for instance types.

Question 10

A machine learning engineer is training a deep learning model for image classification using Amazon SageMaker. The model is a ResNet-50 with 25 million parameters, and the training data is stored in Amazon S3 as 500,000 JPEG images. The engineer wants to minimize training time and cost. They plan to use SageMaker's Pipe mode for data input. Which additional configuration should they implement to maximize I/O throughput?

Accepted Answer

Enable SageMaker Pipe mode with the dataset in S3 and use the RecordIO format.. For large image datasets in S3, SageMaker Pipe mode with RecordIO format maximizes I/O throughput by streaming data and reducing per-image S3 requests. This minimizes startup time and keeps GPUs utilized. Other options either introduce download delays, do not address I/O, or may not resolve the bottleneck efficiently.

Answer

Use a larger instance type with more CPU cores to handle data loading.

Answer

Use Amazon FSx for Lustre as the data source and configure the training job to use File mode.

Answer

Increase the number of epochs to ensure the model sees all data.

Question 11

A machine learning team is using Amazon SageMaker to train a deep learning model for image classification. They are using the SageMaker distributed data parallel library to speed up training across multiple GPU instances. They notice that the training throughput does not scale linearly with the number of instances, and they want to optimize the distributed training configuration. Which two actions should they take to improve scaling efficiency? (Choose two.)

Accepted Answer

Use SageMaker's distributed data parallel library with the 'nccl' backend for GPU-to-GPU communication.. To improve scaling efficiency in SageMaker distributed data parallel training, increasing the batch size per GPU maintains a consistent global batch size and reduces communication overhead. Using the NCCL backend for GPU-to-GPU communication leverages high-speed interconnects for faster all-reduce operations. Both actions address common bottlenecks in distributed training: communication overhead and GPU utilization. Other options either reference invalid parameters or suggest reducing resources, which is not helpful.

Answer

Enable SageMaker's optimized all-reduce algorithm by setting the 'sagemaker_distributed_dataparallel_enabled' parameter to true.

Answer

Reduce the number of instances to match the batch size.

Answer

Set the 'optimize' parameter to 'speed' in the SageMaker distributed data parallel configuration.

Question 12

A team is using Amazon SageMaker Automatic Model Tuning to search for a good XGBoost configuration. They define a hyperparameter as a ContinuousParameter with a range from 0.001 to 1.0, but after several hours the tuning job keeps sampling values clustered near the low end and the best training jobs never explore the upper region. The team wants the search to explore the full range more aggressively in the remaining time. Which change should they make to the tuning job configuration?

Accepted Answer

Switch the strategy from Bayesian to Random and increase max_jobs.. Bayesian search deliberately exploits promising regions, which can look like clustering near one end of a continuous range and can under-explore elsewhere. The Random strategy draws each candidate independently and uniformly across the defined range, so switching strategy plus adding trials increases coverage of the upper values. The other changes affect per-job training length or the optimized metric, not the tuner's sampling behavior.

Answer

Change the hyperparameter to an IntegerParameter with the same numeric bounds.

Answer

Increase the value of the early_stopping_rounds hyperparameter.

Answer

Set the tuning job's objective metric to a different evaluation metric.

Question 13

A team is building a binary classifier for fraudulent transactions using Amazon SageMaker. The dataset has 2 million rows with a 0.4% positive rate. The team wants to maximize recall on the minority class while keeping precision above 0.60, and they need a principled way to set the decision threshold rather than guessing. Which approach best meets these requirements?

Accepted Answer

Use scale_pos_weight to upweight the minority class and select the threshold from a precision-recall curve at the point where precision is at least 0.60. Severe class imbalance makes accuracy and ROC AUC poor guides. Upweighting positives with scale_pos_weight shifts the model toward the minority class, and choosing the operating point on the precision-recall curve where precision is at least 0.60 directly satisfies the business constraint while maximizing recall. This ties threshold selection to the actual metric requirement.

Answer

Train with the default 0.5 threshold and rely on accuracy as the model selection metric

Answer

Downsample the majority class to a 50/50 ratio and use ROC AUC to pick the threshold

Answer

Train with focal loss and set the threshold to the mean predicted probability of the training set

Question 14

A machine learning engineer is using Amazon SageMaker to train a model with the built-in XGBoost algorithm. The engineer wants to improve the model's generalization performance by tuning hyperparameters. Which two hyperparameters should the engineer focus on to control model complexity and prevent overfitting? (Choose two.)

Accepted Answer

max_depth. In XGBoost, max_depth and min_child_weight are structural hyperparameters that directly control the complexity of the trees. max_depth limits how deep each tree can grow, and min_child_weight sets a minimum weight threshold for creating child nodes. Both are effective at preventing overfitting by constraining the model's ability to fit noise in the training data. Other parameters like eta, subsample, and num_round also affect overfitting but are not primarily used to control model complexity.

Answer

subsample

Answer

num_round

Answer

eta

Question 15

A data scientist is training a gradient boosting model using Amazon SageMaker's built-in XGBoost algorithm on a dataset with 1 million rows and 200 features. The training job is taking longer than expected, and the data scientist wants to reduce training time without significantly sacrificing model accuracy. The dataset is stored in Amazon S3 in CSV format. Which action is MOST likely to reduce training time?

Accepted Answer

Convert the training data to the RecordIO Protobuf format and enable the use of additional instances for distributed training.. For large datasets, distributed training with XGBoost in SageMaker can dramatically reduce training time. Converting data to RecordIO Protobuf enables efficient sharding across multiple instances. This method preserves model accuracy because XGBoost supports distributed training natively. Other options either increase training time or risk accuracy loss.

Answer

Reduce the number of features by applying PCA before training.

Answer

Use a smaller instance type with more memory to fit the entire dataset in memory.

Answer

Increase the number of epochs and reduce the learning rate to improve convergence.