MLS-C01 Modeling Practice Test 18 — 15 Questions

Question 1

A data scientist is using Amazon SageMaker to train a deep learning model with a custom training script. The script uses TensorFlow and requires a specific version of a Python library that is not included in the default SageMaker TensorFlow container. The scientist needs to ensure the library is available during training. What is the MOST efficient way to accomplish this?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use the SageMaker TensorFlow estimator with a requirements.txt file in the source directory.. Using a requirements.txt file in the source directory is the most efficient method because SageMaker automatically installs the specified packages in the training container. It avoids the overhead of building a custom container and ensures the library is available for the training script. Other methods either do not affect the training environment or are unnecessarily complex.

Answer

Create a custom Docker container with the required library and push it to Amazon ECR.

Answer

Use Amazon SageMaker Studio and install the library in the notebook kernel.

Answer

Pass the library as a pip install command in the entry point script before importing it.

Question 2

A financial institution is building a model to predict loan default. The dataset is highly imbalanced, with only 2% of loans defaulting. The team wants to use Amazon SageMaker to train a binary classification model and ensure that the model performs well on the minority class. Which evaluation metric should they optimize during training?

Accepted Answer

F1 score. The correct answer is F1 score because it balances precision and recall, making it effective for imbalanced classification where the minority class is of interest. Accuracy is misleading, AUC-ROC may not capture minority class performance adequately, and RMSE is for regression.

Answer

Root Mean Squared Error (RMSE)

Answer

AUC-ROC

Answer

Accuracy

Question 3

A machine learning specialist is using Amazon SageMaker to train a deep learning model for image segmentation. The model is trained on a large dataset of high-resolution images. During training, the specialist notices that the GPU memory is exhausted, causing the training job to fail. The specialist wants to reduce GPU memory usage without significantly impacting model accuracy. Which two strategies should the specialist use? (Choose two.)

Accepted Answer

Enable mixed precision training by using AMP (Automatic Mixed Precision) with float16.. Reducing batch size with gradient accumulation lowers peak memory by processing fewer samples at once while maintaining effective batch size. Mixed precision training cuts memory by using float16 for activations and gradients. Both are software-level optimizations that reduce memory without sacrificing accuracy, unlike increasing epochs or downscaling images.

Answer

Increase the number of epochs to allow the model to converge with less memory per step.

Answer

Reduce the image resolution and crop images to smaller sizes during training.

Answer

Use a larger instance type with more GPU memory, such as an ml.p4d.24xlarge.

Question 4

A data scientist is training a model using Amazon SageMaker's built-in XGBoost algorithm on a dataset with both numerical and categorical features. The model's performance is poor, and the data scientist suspects that the categorical features are not being handled properly. Which two actions should the data scientist take to improve the model's performance? (Choose two.)

Accepted Answer

Convert categorical features to numerical using label encoding.. SageMaker's built-in XGBoost requires numerical input, so categorical features must be encoded. One-hot encoding and label encoding are both valid methods to convert categorical variables into numerical form. One-hot encoding is suitable for low-cardinality features, while label encoding can be used for high-cardinality features. Both actions directly address the encoding problem and can improve model performance. The other options either rely on unsupported functionality or do not resolve the encoding issue.

Answer

Use SageMaker Automatic Model Tuning to optimize the 'max_depth' hyperparameter.

Answer

Use the 'cat' feature type in the SageMaker XGBoost algorithm to automatically handle categorical features.

Answer

Increase the number of boosting rounds to allow the model to learn the categorical patterns.

Question 5

A data scientist is using Amazon SageMaker to train a deep learning model with a custom training script. The script uses TensorFlow and requires a specific version of a Python library that is not included in the default SageMaker TensorFlow container. The scientist needs to ensure the library is available during training. What is the MOST efficient way to accomplish this?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use Pipe mode with the built-in TensorFlow estimator and set the 'input_mode' parameter to 'Pipe'.. Pipe mode streams data from Amazon S3 directly to the training algorithm, avoiding the need to download the full dataset to local storage. This reduces startup latency and disk requirements, making it ideal for large datasets. For a 500 GB dataset, Pipe mode minimizes training time and cost by feeding data on the fly. Other modes either incur download overhead or are not optimized for sequential large-scale reading.

Answer

Use File mode but pre-download the data to Amazon EBS and attach the volume to the training instance.

Answer

Use FastFile mode with a smaller instance type to save cost.

Answer

Use File mode with a large instance type and increase the number of epochs.

Question 6

A machine learning engineer is using Amazon SageMaker to train a model on a dataset with 100 features. After training, the model's performance is good, but inference latency is high. The engineer wants to reduce the number of features without significantly sacrificing accuracy. Which technique should be used?

Accepted Answer

Principal Component Analysis (PCA). The correct answer is PCA because it reduces dimensionality by projecting data onto principal components, which can lower inference latency while retaining most variance. RFE selects features but may not reduce as effectively, t-SNE is for visualization, and autoencoders add complexity.

Answer

t-SNE

Answer

Recursive Feature Elimination (RFE)

Answer

Autoencoders

Question 7

A data scientist is using Amazon SageMaker to train a deep learning model with a custom training script. The script uses TensorFlow and requires a specific version of a Python library that is not included in the default SageMaker TensorFlow container. The scientist needs to ensure the library is available during training. What is the MOST efficient way to accomplish this?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use the average silhouette score computed on the test data for each k and choose the k with the highest score.. The average silhouette score provides a quantitative measure of cluster quality that balances cohesion and separation. It can be computed for different k values, and the k with the highest score is selected. This method is objective and works well even when the elbow is not clear, making it more reliable than the elbow method or training loss.

Answer

Use SageMaker Automatic Model Tuning to minimize the 'test:msd' metric and select the best k.

Answer

Use the elbow method by plotting the within-cluster sum of squares (WCSS) against k and looking for an elbow.

Answer

Train a separate model for each k and select the one with the lowest training loss.

Question 8

A team trains a SageMaker built-in Linear Learner model for a regression task predicting delivery time in minutes. After training, the model's validation root mean squared error is 480 minutes, far worse than a baseline mean predictor. The feature set includes raw timestamps, delivery distance in kilometers, package weight, and a high-cardinality carrier ID. Which change is MOST likely to improve the model?

Accepted Answer

Set the predictor_type to regressor and enable the normalize_label and normalize_features options in the Linear Learner hyperparameters.. Linear Learner fits coefficients on a shared scale and applies one regularization parameter across all dimensions. When one feature is orders of magnitude larger than the others, the effective learning rate is wrong for most coefficients. Enabling feature and label normalization restores balanced conditioning, which typically transforms RMSE from useless to competitive for this kind of tabular regression.

Answer

Replace Linear Learner with the built-in K-Means algorithm and cluster deliveries by duration.

Answer

Increase the mini_batch_size and epochs so the optimizer has more updates to find the global minimum.

Answer

Switch the problem type to binary classification and binarize the delivery time at the median.

Question 9

A machine learning engineer is training a binary classifier on a dataset with 1,000,000 examples and 500 features. They use Amazon SageMaker's built-in Linear Learner algorithm in a training job with two ml.c5.4xlarge instances. After training, they notice that the model's validation accuracy is significantly lower than the training accuracy, and the training loss continues to decrease while validation loss starts to increase after 10 epochs. Which technique should they implement to address this issue?

Accepted Answer

Enable early stopping in the Linear Learner hyperparameters.. The model is overfitting: training loss decreases while validation loss increases. Early stopping is a regularization technique that halts training when validation performance stops improving, preventing the model from memorizing noise. In SageMaker Linear Learner, early stopping is controlled via hyperparameters like early_stopping_patience and early_stopping_tolerance, making it the appropriate solution.

Answer

Reduce the number of features by performing PCA before training.

Answer

Increase the number of epochs to 100.

Answer

Add more layers to the model by switching to a deep neural network.

Question 10

A data scientist is using Amazon SageMaker to train a deep learning model with a custom training script. The script uses TensorFlow and requires a specific version of a Python library that is not included in the default SageMaker TensorFlow container. The scientist needs to ensure the library is available during training. What is the MOST efficient way to accomplish this?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Increase the model complexity (e.g., use a deeper neural network or higher-degree polynomial).. The correct answers are increasing model complexity and reducing regularization strength. Underfitting occurs when the model is too simple or overly constrained, so enhancing capacity or relaxing constraints allows it to better fit the data. Adding data or increasing regularization would not help and could worsen the issue.

Answer

Perform feature engineering to add more relevant features.

Answer

Add more training data.

Answer

Increase regularization strength.

Question 11

A machine learning engineer has trained a SageMaker model and now needs to evaluate its performance on a held-out test set. The engineer wants to use the SageMaker Python SDK to obtain standard regression metrics such as mean squared error and root mean squared error without writing custom metric code. Which approach should the engineer take?

Accepted Answer

Use the SageMaker SDK's model.evaluate or the analytics helpers to compute regression metrics against a labeled dataset.. The SageMaker Python SDK includes evaluation support that takes a trained model or predictor plus a labeled dataset and returns task-appropriate metrics, so regression scores such as MSE and RMSE are produced without hand-written formulas. Deploying an endpoint and scoring records manually works but requires custom metric computation and adds hosting cost. Retraining or parsing artifacts does not evaluate the finished model on a held-out test set and typically relies on validation-split metrics instead.

Answer

Read the model artifact from Amazon S3 and parse the serialized model file to extract stored evaluation scores.

Answer

Re-run the training job with a different objective metric so the final logs contain the test metrics.

Answer

Call the model's deploy method and invoke the endpoint, then compute metrics manually in the notebook.

Question 12

A machine learning engineer is training a model on Amazon SageMaker and wants to improve model generalization by reducing overfitting. The engineer is using a built-in XGBoost algorithm. Which two hyperparameters should the engineer adjust to reduce overfitting? (Choose two.)

Accepted Answer

Decrease the value of max_depth.. To reduce overfitting in XGBoost, one can increase regularization (lambda) and decrease model complexity (max_depth). Increasing lambda penalizes large weights, while decreasing max_depth limits tree depth. Increasing eta or subsample would likely worsen overfitting, and changing the objective is unrelated to regularization.

Answer

Set the objective to reg:logistic.

Answer

Increase the value of eta (learning rate).

Answer

Increase the value of subsample.

Question 13

A data scientist is using Amazon SageMaker to train a deep learning model with a custom training script. The script uses TensorFlow and requires a specific version of a Python library that is not included in the default SageMaker TensorFlow container. The scientist needs to ensure the library is available during training. What is the MOST efficient way to accomplish this?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Change the tuning strategy from Bayesian to Random and widen the hyperparameter ranges.. The tuning job likely used Bayesian optimization, which can focus on promising regions and miss others, especially with limited jobs. Random search samples independently from the defined ranges, ensuring broad coverage. Widening the ranges further expands the search space. Together, these changes prevent repeated trials and improve the chance of finding better hyperparameters.

Answer

Set the early stopping type to 'Auto' to terminate poorly performing trials.

Answer

Increase the number of epochs for each training job to allow the model to better differentiate hyperparameters.

Answer

Increase max_parallel_jobs to 20 to allow more concurrent trials.

Question 14

A data scientist is using Amazon SageMaker to train a deep learning model with a custom training script. The script uses TensorFlow and requires a specific version of a Python library that is not included in the default SageMaker TensorFlow container. The scientist needs to ensure the library is available during training. What is the MOST efficient way to accomplish this?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Apply feature scaling to the numerical features before clustering.. K-means clustering relies on distance calculations, so features with larger scales can disproportionately influence cluster formation. Standardizing the numerical features to have equal variance ensures that all features contribute equally, often leading to more distinct and meaningful clusters. This preprocessing step directly addresses the overlapping issue observed in the PCA visualization.

Answer

Increase the maximum number of iterations for the k-means algorithm.

Answer

Increase the number of clusters to 10 to capture more granularity.

Answer

Use a different dimensionality reduction technique like t-SNE for visualization.

Question 15

A data scientist is tuning a binary classification model using Amazon SageMaker automatic model tuning (AMT). The objective metric is 'validation:auc', and the tuning job has completed 50 training jobs. The data scientist notices that the best training job achieved an AUC of 0.85, but the average AUC across all jobs is 0.70, with high variance. The scientist wants to improve the tuning job's ability to find better hyperparameters without increasing the number of training jobs. Which strategy should the scientist use?

Accepted Answer

Configure the tuning job to use the 'Hyperband' strategy and define a resource limit per job.. Hyperband is a bandit-based strategy that dynamically allocates resources to promising configurations and stops underperforming ones early. In SageMaker AMT, using Hyperband with a resource limit allows the tuning job to evaluate more hyperparameter combinations within the same total training budget, effectively improving the search's ability to find better hyperparameters without increasing the number of jobs.

Answer

Use a Bayesian optimization strategy with a Gaussian process and define a wider hyperparameter range.

Answer

Enable early stopping for the tuning job and set a larger max_wait time.

Answer

Switch the hyperparameter tuning strategy from Bayesian to Random search.