MLS-C01 Modeling Practice Test 51 — 15 Questions

Question 1

A data scientist is training a deep learning model using Amazon SageMaker and wants to use distributed training to speed up the process. The model is a large transformer with billions of parameters, and the training data is stored in Amazon S3. The data scientist decides to use SageMaker's distributed data parallel library. Which of the following is a key consideration when using this library?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The library can be enabled by specifying the distribution parameter with 'smdistributed' and 'dataparallel' in the estimator.. SageMaker's distributed data parallel library is enabled through the estimator's distribution parameter, specifying 'smdistributed' and 'dataparallel'. It does not require Horovod, does not auto-shard data, and while EFA is beneficial, it is not strictly required for all instances.

Answer

The library requires the training script to use the Horovod API for communication.

Answer

The library requires the use of Elastic Fabric Adapter (EFA) for all instance types.

Answer

The library automatically shards the training data across instances, so no additional data partitioning is needed.

Question 2

A data scientist is preparing a dataset for training a linear learner model in Amazon SageMaker. The dataset contains a numerical feature with a highly skewed distribution and several categorical features with high cardinality. The team wants to improve model performance and reduce training time. Which two preprocessing steps should the data scientist perform? (Choose two.)

Accepted Answer

Use target encoding for the high-cardinality categorical features.. The skewed numerical feature can distort linear model training, so a log transform makes it more Gaussian-like. High-cardinality categorical features cause dimensionality issues with one-hot encoding, so target encoding provides a compact representation. These two steps address the specific challenges and can improve both performance and training efficiency.

Answer

One-hot encode the high-cardinality categorical features.

Answer

Standardize all numerical features to have zero mean and unit variance.

Answer

Remove all categorical features to simplify the model.

Question 3

A machine learning engineer is using Amazon SageMaker to train a deep learning model for medical image segmentation. The dataset contains 10,000 images with pixel-level masks. The engineer wants to use the built-in Semantic Segmentation algorithm but finds that the algorithm expects a specific input format. The images and masks are stored in Amazon S3 as PNG files. Which preprocessing step is required to prepare the data for the SageMaker Semantic Segmentation algorithm?

Accepted Answer

Create a JSON label map file that maps class IDs to class names, and organize images and masks in separate folders.. The SageMaker Semantic Segmentation algorithm expects a label map JSON file that defines the mapping between class IDs and class names, including a background class. Images and masks must be in PNG format and organized in separate folders. The training job then references the S3 paths for images and masks. Creating this label map and structure is the essential preprocessing step before launching training.

Answer

Flatten each image and mask into a CSV file with one row per pixel, including x, y coordinates and class labels.

Answer

Convert the images and masks to RecordIO protobuf format using the im2rec tool.

Answer

Convert all images to grayscale and resize them to 256x256 pixels, then store as TFRecord files.

Question 4

A data scientist is training a model on Amazon SageMaker using the built-in Object2Vec algorithm to learn embeddings for a recommendation task. The training dataset consists of user-item interaction pairs. After training, the data scientist wants to evaluate the quality of the learned embeddings. Which evaluation method is most appropriate for assessing the embeddings?

Accepted Answer

Use the embeddings as features to train a downstream classifier and evaluate its accuracy on a held-out set.. Extrinsic evaluation by training a downstream classifier on the embeddings and measuring its accuracy on held-out data is a robust way to assess embedding quality. It tests whether the embeddings capture information that generalizes to new data, which is the ultimate goal for recommendation tasks. Intrinsic metrics like loss or visualization are less directly tied to practical utility.

Answer

Compute the average cosine similarity between random user-item pairs.

Answer

Perform a t-SNE visualization of the embeddings and visually inspect for clusters.

Answer

Measure the training loss of the Object2Vec model on the test dataset.

Question 5

A machine learning specialist is using Amazon SageMaker to train a model with the built-in k-means algorithm. The dataset contains numerical features with varying scales, and the specialist wants to ensure that the clustering is not dominated by features with larger magnitudes. Which preprocessing step should the specialist perform before training?

Accepted Answer

Normalize the features to have zero mean and unit variance.. k-means relies on distance calculations, so features with larger scales can dominate the clustering. Normalizing features to zero mean and unit variance ensures equal contribution from all features, leading to more meaningful clusters. This is a standard and necessary preprocessing step when features have different units or ranges.

Answer

Convert all features to categorical bins.

Answer

Apply one-hot encoding to all features.

Answer

Apply principal component analysis (PCA) to reduce dimensionality.

Question 6

A data scientist is building a time-series forecasting model using Amazon SageMaker. The dataset contains daily sales data for 500 products over two years, with strong weekly and yearly seasonality. The data scientist wants to use the built-in DeepAR algorithm. Which data format is required for the training input?

Accepted Answer

A JSON Lines file where each line is a JSON object containing a start timestamp, a target array, and optional dynamic features.. DeepAR expects training data in JSON Lines format, where each line represents a single time series with a 'start' timestamp, a 'target' array of values, and optional 'dynamic_feat' for covariates. This format allows the algorithm to process multiple related time series and learn cross-series patterns. The data scientist must convert the dataset into this format before training.

Answer

A CSV file where each row represents a time step and columns are the target and related features.

Answer

A RecordIO protobuf file with each record containing a sequence of floats and a label.

Answer

A Parquet file with columns for item_id, timestamp, and sales, sorted by timestamp.

Question 7

A data scientist is using Amazon SageMaker to train a model with the built-in DeepAR algorithm for time series forecasting. The dataset contains 100 related time series, each with 2 years of daily data. The data scientist notices that the model performs well on most series but poorly on a few series with intermittent demand (many zero values). Which modification to the training data or algorithm configuration is most likely to improve performance on the intermittent series?

Accepted Answer

Set the likelihood parameter to 'negative-binomial' to better model count data with zeros.. Intermittent demand series are characterized by many zeros and occasional positive counts. DeepAR's default likelihood may not handle this well. Switching to the negative binomial likelihood allows the model to account for overdispersion and zero inflation, leading to better forecasts for these series. This is a targeted configuration change that addresses the underlying data distribution.

Answer

Increase the number of epochs to allow the model to better learn the zero patterns.

Answer

Remove the intermittent series from the training dataset to prevent them from degrading overall model performance.

Answer

Increase the context_length hyperparameter to include more historical data.

Question 8

A team is training a SageMaker built-in linear learner model for a binary classification problem with highly imbalanced classes. They set the predictor_type to binary_classifier and enable auto model tuning. After training, the model's accuracy is high but recall on the minority class is near zero. Which change should they make to the linear learner hyperparameters to improve minority-class recall?

Accepted Answer

Set the positive_example_weight_mult parameter to a value greater than 1.. The linear learner algorithm supports positive_example_weight_mult, which multiplies the weight of positive examples in the logistic loss. When the positive class is rare, the unweighted objective drives the boundary toward the majority class, yielding high accuracy but near-zero recall. Raising this multiplier makes positive errors more expensive, shifting the boundary and improving minority-class recall.

Answer

Increase the value of the learning_rate hyperparameter.

Answer

Increase the value of the l1 regularization parameter.

Answer

Switch the predictor_type to regressor and round the predictions.

Question 9

A data scientist is using Amazon SageMaker to train a deep learning model with the built-in Object Detection algorithm. The training dataset contains images with bounding boxes for multiple object classes. The scientist notices that the model performs well on the training set but poorly on the validation set, and the validation loss increases after a few epochs. Which action should the scientist take to improve generalization?

Accepted Answer

Enable data augmentation and increase the weight decay hyperparameter.. Data augmentation increases the diversity of training data without collecting new images, helping the model generalize better. Increasing weight decay applies stronger L2 regularization, which penalizes complex models. Both are standard techniques to combat overfitting, as indicated by the rising validation loss and large train-validation gap.

Answer

Reduce the number of training epochs and increase the number of classes.

Answer

Use a larger pre-trained model and freeze all layers.

Answer

Increase the learning rate and reduce the batch size.

Question 10

A data scientist is training a linear regression model using Amazon SageMaker's Linear Learner algorithm on a dataset with 50 features. After training, the model's predictions on the validation set are consistently off by a large margin. The scientist suspects that the features have vastly different scales. Which preprocessing step should be applied to the training data to improve the model's performance?

Accepted Answer

Normalize the features to have zero mean and unit variance.. Linear models like Linear Learner rely on gradient descent, which converges faster and more reliably when features are on similar scales. Normalizing features to zero mean and unit variance ensures that the optimization landscape is well-conditioned, preventing features with larger scales from dominating the gradient updates. This leads to better model performance and more accurate predictions.

Answer

Apply one-hot encoding to all numerical features.

Answer

Remove features with low variance.

Answer

Apply principal component analysis (PCA) to reduce dimensionality.

Question 11

A data scientist is using Amazon SageMaker to train a model with the built-in K-Means algorithm. They have a dataset with 10,000 observations and want to segment them into 5 clusters. After training, they notice that the clusters are highly imbalanced, with one cluster containing over 90% of the data points. Which action should they take to improve cluster quality?

Accepted Answer

Standardize the features before training.. K-Means clustering relies on distance calculations, which are affected by feature scales. Unscaled features can dominate, causing skewed clusters. Standardizing features ensures each feature contributes equally, leading to more balanced and meaningful clusters. Other options do not directly address the scaling issue and may not resolve the imbalance.

Answer

Use a different distance metric such as Manhattan distance.

Answer

Increase the number of clusters to 10.

Answer

Reduce the number of features by removing less important ones.

Question 12

A financial institution is building a fraud detection model using Amazon SageMaker. The dataset is highly imbalanced, with only 0.5% of transactions being fraudulent. The team wants to maximize the detection of fraudulent transactions while keeping false positives at an acceptable level. They decide to use the SageMaker built-in XGBoost algorithm. Which evaluation metric and corresponding hyperparameter tuning strategy should they use to achieve their goal?

Accepted Answer

Optimize for F1 score and tune scale_pos_weight along with max_depth.. In fraud detection with extreme imbalance, the F1 score provides a balance between precision and recall, which is crucial for maximizing fraud detection while keeping false positives acceptable. XGBoost's scale_pos_weight hyperparameter adjusts the loss function to give more weight to the positive class, effectively addressing imbalance. Tuning max_depth controls model complexity, preventing overfitting to the minority class. This strategy is widely recommended for imbalanced binary classification.

Answer

Optimize for precision and tune subsample and colsample_bytree.

Answer

Optimize for accuracy and tune the number of rounds and learning rate.

Answer

Optimize for area under the ROC curve (AUC) and tune max_depth and eta.

Question 13

A machine learning engineer is training a SageMaker built-in k-means model to segment customers based on behavioral features. The engineer wants to choose the number of clusters and evaluate cluster quality. The training data is unlabeled. Which approach is MOST appropriate for selecting the number of clusters in this scenario?

Accepted Answer

Train models with different k values and compare the average distance of each point to its assigned cluster centroid.. Because clustering is unsupervised, cluster-count selection must rely on intrinsic quality measures. The within-cluster sum of squared distances to centroids is the objective k-means minimizes, so comparing it across several k values, typically with an elbow plot, shows the point of diminishing returns. This approach requires no labels and is directly supported by the SageMaker built-in k-means training output.

Answer

Use the F1 score computed on a held-out validation set to pick the best k.

Answer

Use the model's training accuracy metric reported in the SageMaker training logs.

Answer

Increase k until the number of clusters equals the number of training examples, ensuring every point is its own cluster.

Question 14

A machine learning engineer is training a gradient boosting model with Amazon SageMaker's built-in XGBoost algorithm on a tabular dataset where the target is a continuous value with a long right tail, and the team suspects that a few extreme outliers are distorting the loss. They want the training objective itself to be less sensitive to those extreme residuals without changing the input data. Which SageMaker built-in XGBoost hyperparameter should they tune to achieve this?

Accepted Answer

objective set to reg:pseudohubererror. The built-in XGBoost container exposes objective as a hyperparameter, and reg:pseudohubererror applies a robust pseudo-Huber loss whose gradients are bounded for large residuals. That directly reduces the leverage of extreme target values during boosting. Reporting metrics, class weighting, or depth limits do not alter the fundamental squared-error sensitivity that the scenario attributes to the outliers.

Answer

max_depth set to a small value

Answer

eval_metric set to rmse

Answer

scale_pos_weight set to a large value

Question 15

A machine learning engineer is training a gradient boosting model with the Amazon SageMaker built-in XGBoost algorithm on a tabular dataset with 40 features and 500,000 rows. During training, the model achieves low training error but high validation error, and the gap widens as training progresses. The engineer wants to reduce overfitting without collecting more data. Which combination of hyperparameter adjustments should the engineer make?

Accepted Answer

Decrease max_depth and increase lambda. When training error is low but validation error is high and growing, the model has too much capacity relative to the data. Reducing max_depth constrains tree complexity, and increasing lambda penalizes large leaf weights, both lowering variance. The other choices either increase capacity or extend training in ways that sustain or worsen the train-validation gap.

Answer

Increase alpha and increase num_round

Answer

Increase max_depth and increase eta

Answer

Increase num_round and decrease subsample