MLS-C01 Modeling Practice Test 18 — 15 Questions

Question 1

Which TWO actions can help reduce overfitting in a neural network? (Choose 2.)

Accepted Answer

Apply L1 or L2 regularization.. Options C and E are correct because L1/L2 regularization penalizes large weights to prevent overfitting, and dropout randomly deactivates neurons to reduce co-adaptation. Option A is incorrect because adding layers increases model complexity, which can worsen overfitting. Option B is incorrect because decreasing the learning rate only affects training speed, not overfitting. Option D is incorrect because, although increasing dataset size can help reduce overfitting, it is not one of the two actions specified; L1/L2 regularization and dropout are the correct choices.

Answer

Increase the number of layers.

Answer

Decrease the learning rate.

Answer

Increase the training dataset size.

Question 2

Which THREE evaluation metrics are appropriate for a multi-class classification problem? (Choose 3.)

Accepted Answer

Confusion matrix.. Confusion matrix (A) is appropriate because it provides per-class performance metrics (TP, FP, FN, TN) for each class, which is essential for multi-class evaluation. Accuracy (B) is appropriate as it measures overall correctness across all classes, a common and intuitive metric for multi-class problems. F1 score with macro or micro averaging (E) is appropriate because macro averaging computes F1 per class and averages them equally, while micro averaging aggregates contributions across all classes, both suitable for multi-class. Mean squared error (C) is incorrect; it is a regression metric not used for classification. Precision-recall curve (D) is typically used for binary classification, not standard for multi-class without extensions.

Answer

Mean squared error.

Answer

Precision-recall curve.

Question 3

Which TWO techniques are used to handle missing values in a dataset before training? (Choose 2.)

Accepted Answer

Mean or median imputation.. Mean or median imputation is a common method to fill missing values with a central tendency measure. Option C is correct because removing rows or columns with missing values is a valid approach, especially when the missing data is minimal. Option B (min-max scaling) is for normalizing numerical features, not for missing values. Option D (one-hot encoding) is for converting categorical variables into numerical format. Option E (PCA) is for dimensionality reduction, not missing value handling.

Answer

Min-max scaling.

Answer

One-hot encoding.

Answer

Principal component analysis (PCA).

Question 4

A data scientist is training a binary classification model on an imbalanced dataset where the positive class represents only 5% of the data. The model currently achieves 95% accuracy but only 10% recall on the positive class. Which metric should the scientist focus on to improve the model's ability to detect the positive class?

Accepted Answer

Recall. (Recall) is the correct focus because recall measures the proportion of actual positive cases correctly identified. With only 10% recall, the model is missing most positive cases despite high accuracy due to class imbalance. Improving recall directly addresses the goal of detecting the positive class. Option B (Accuracy) is misleading in imbalanced datasets as it can be high even if the model predicts all negatives. Option C (Precision) measures the proportion of positive predictions that are correct, which may not increase recall. Option D (AUC-ROC) is a global metric that may not reflect improvements in recall specifically.

Answer

Accuracy

Answer

Precision

Answer

AUC-ROC

Question 5

A machine learning engineer is using Amazon SageMaker to train a model. The training job fails with an out-of-memory error. The training data size is 10 GB and the instance is ml.m5.xlarge (16 GB memory). Which change is MOST likely to resolve the issue without increasing cost?

Accepted Answer

Reduce the batch size in the training script.. Many algorithms allow you to set a batch size, and reducing it lowers memory usage. Option B is wrong because changing to GPU may not help and could increase cost. Option C is wrong because increasing instance type increases cost. Option D is wrong because decreasing the dataset size may lose information.

Answer

Switch to a GPU instance like p3.2xlarge.

Answer

Use a larger instance type like ml.m5.4xlarge.

Answer

Decrease the training dataset size.

Question 6

An e-commerce company wants to build a recommendation system. They have user-item interaction data (clicks, purchases) and user demographic data. The goal is to recommend items that a user is likely to purchase. Which approach should be used?

Accepted Answer

Factorization Machines using user-item interactions and user features.. Factorization Machines are designed for high-dimensional sparse data and can effectively combine user-item interactions with side features like user demographics. Option A is wrong because linear regression is not suitable for implicit feedback or modeling interactions. Option B is wrong because collaborative filtering (e.g., matrix factorization) does not naturally incorporate user demographic features. Option D is wrong because content-based filtering only uses item features and ignores user-item interaction patterns.

Answer

Linear regression on user and item features.

Answer

Collaborative filtering using matrix factorization.

Answer

Content-based filtering using item features.

Question 7

A data scientist is training a deep learning model for image classification using TensorFlow on Amazon SageMaker. The model trains slowly, and the GPU utilization is below 20%. Which action will MOST effectively increase GPU utilization and reduce training time?

Accepted Answer

Increase the batch size to better saturate the GPU.. Increasing batch size provides more work per GPU step, improving utilization. Option A is wrong because reducing dataset size does not directly improve GPU utilization and may lead to underfitting. Option C is wrong because switching to CPU would be slower. Option D is wrong because decreasing batch size reduces the work per step, lowering GPU utilization and increasing training time.

Answer

Reduce the training dataset size.

Answer

Switch to a CPU-only instance.

Answer

Decrease the batch size to reduce memory pressure.

Question 8

A company uses Amazon SageMaker to train a model and wants to track metrics like loss and accuracy in real-time. Which SageMaker feature should be used?

Accepted Answer

SageMaker metrics and CloudWatch dashboards. SageMaker's built-in metrics integration with CloudWatch allows real-time tracking of training metrics such as loss and accuracy. Option A (SageMaker Model Monitor) is for monitoring inference quality and data drift after deployment, not real-time training metrics. Option C (SageMaker Experiments) is for organizing, tracking, and comparing different experiment runs, but it does not provide real-time streaming of metrics during training. Option D (SageMaker Debugger) is used for debugging training issues like gradient anomalies, not for real-time metric tracking.

Answer

SageMaker Model Monitor

Answer

SageMaker Experiments

Answer

SageMaker Debugger

Question 9

A data scientist is using Amazon SageMaker to build a text classification model. The dataset has 100,000 labeled samples and 20 classes. The scientist wants to use a pre-trained BERT model and fine-tune it. Which approach is MOST cost-effective?

Accepted Answer

Fine-tune a pre-trained BERT-base model using a GPU instance.. Fine-tuning a pre-trained BERT-base model on a GPU instance is the most cost-effective approach. Pre-trained BERT models already capture general language features, so fine-tuning requires less computation and data compared to training from scratch. BERT-base is smaller than BERT-large, reducing cost while still being effective for text classification. Option A is wrong because training BERT from scratch is extremely expensive and unnecessary. Option C is wrong because BERT-large has higher cost and may overfit given the dataset size. Option D is wrong because CNN models from scratch would require more hyperparameter tuning and may not achieve comparable accuracy to a fine-tuned transformer.

Answer

Train a BERT model from scratch using a larger instance.

Answer

Use a pre-trained BERT-large model with a larger instance.

Answer

Train a CNN model from scratch using CPU instances.

Question 10

A company is deploying a machine learning model for real-time fraud detection. The model must have low latency (under 100 ms) and high throughput. The model is an ensemble of 5 gradient boosted trees (XGBoost), each 200 MB. Which deployment strategy is MOST suitable?

Accepted Answer

Use SageMaker multi-model endpoint on a compute-optimized instance.. A multi-model endpoint on a compute-optimized instance allows loading multiple models dynamically, reducing cost and latency compared to separate endpoints. Option A is wrong because invoking each model sequentially with Lambda would incur overhead and cold start latency, exceeding the 100 ms requirement. Option B is wrong because deploying each model as a separate SageMaker endpoint would require managing multiple endpoints, increasing cost and complexity without benefiting from model sharing on a single instance. Option C is wrong because batch processing is not suitable for real-time inference; GPU instances are overkill for tree-based models and add latency.

Answer

Use AWS Lambda to invoke each model sequentially.

Answer

Deploy each model as a separate SageMaker endpoint and use a load balancer.

Answer

Deploy the ensemble on a single GPU instance with large batch processing.

Question 11

Which TWO of the following are best practices for hyperparameter tuning using Amazon SageMaker? (Choose 2)

Accepted Answer

Use early stopping to terminate poorly performing training jobs.. Early stopping terminates poorly performing training jobs, saving time and compute resources. Option E is correct because automatic model tuning with Bayesian optimization efficiently searches the hyperparameter space by focusing on promising regions. Option A is incorrect because grid search is exhaustive and computationally expensive, not a best practice. Option C is incorrect because tuning all algorithm hyperparameters can be unnecessary and inefficient; only relevant hyperparameters should be tuned. Option D is incorrect because while a larger dataset may improve model accuracy, it does not directly improve tuning efficiency; hyperparameter tuning optimizes the learning process, not data size.

Answer

Use grid search to exhaustively explore all combinations.

Answer

Include all algorithm hyperparameters in the tuning job.

Answer

Use a larger training dataset to improve tuning results.

Question 12

Which THREE of the following are valid strategies to reduce overfitting in a deep neural network? (Choose 3)

Accepted Answer

Use early stopping.. Early stopping halts training when validation performance degrades, preventing overfitting. Option D is correct because L2 regularization adds a penalty on large weights, discouraging complexity. Option E is correct because dropout randomly drops neurons during training, reducing co-adaptation. Option A is wrong because adding more layers increases model capacity, which exacerbates overfitting. Option C is wrong because a higher learning rate can cause the loss to diverge and does not directly address overfitting.

Answer

Increase the number of layers.

Answer

Increase the learning rate.

Question 13

Which TWO of the following are appropriate use cases for using Amazon SageMaker BlazingText? (Choose 2)

Accepted Answer

Text classification using supervised learning.. Amazon SageMaker BlazingText supports text classification using supervised learning. Option C is correct because BlazingText can learn word embeddings (e.g., Word2Vec) from large text corpora. Option B is incorrect because time series forecasting is not a capability of BlazingText; it is suited for NLP tasks. Option D is incorrect because BlazingText does not support image classification—that would require a different service or algorithm. Option E is incorrect because sequence-to-sequence translation is not supported by BlazingText; it is designed for word-level embeddings and text classification.

Answer

Time series forecasting.

Answer

Classifying images.

Answer

Sequence-to-sequence translation.

Question 14

A financial services company uses Amazon SageMaker to train a model for credit risk prediction. The dataset contains 500 features and 1 million records. The target variable is binary with 20% default rate. The data scientist uses a gradient boosting algorithm (XGBoost) with default hyperparameters. After training, the model achieves 95% accuracy, but the precision for the default class is only 30%, and recall is 15%. The business requires at least 50% recall and 40% precision for the default class. The data scientist tries to adjust the decision threshold, but this does not simultaneously meet both targets. The scientist suspects that the model is not learning the default patterns well. The company also has a large dataset of unlabeled transactions that could be used. Which action should the data scientist take to improve the model?

Accepted Answer

Use the unlabeled data for semi-supervised learning with pseudo-labeling.. Using unlabeled data for pseudo-labeling can help the model learn patterns of the minority class by generating additional training examples, which is especially beneficial when the labeled dataset is imbalanced. Option A is incorrect because PCA reduces dimensionality but does not address class imbalance and may discard features important for the default class. Option C is incorrect because increasing the learning rate can cause the model to overshoot optimal minima and may lead to overfitting, not improving recall and precision for the minority class. Option D is incorrect because feature selection reduces the number of features but does not directly address class imbalance; it could even remove features that are relevant for predicting defaults.

Answer

Apply PCA to reduce dimensionality and noise.

Answer

Increase the learning rate to accelerate convergence.

Answer

Reduce the number of features using feature selection to simplify the model.

Question 15

A healthcare company is building a model to predict patient readmission within 30 days of discharge. The dataset includes 10,000 patient records with 200 features, including lab results, demographics, and historical admissions. The target variable is highly imbalanced: only 8% of patients are readmitted. The data scientist splits the data into 80% training and 20% test sets, ensuring the same proportion of readmissions in each. The scientist trains a logistic regression model and a random forest model. The logistic regression achieves 92% accuracy but recall of 10% for the readmitted class. The random forest achieves 90% accuracy but recall of 25%. The business requirement is to achieve at least 60% recall for readmissions while maintaining reasonable precision. The scientist also has access to a large collection of unlabeled patient records from other hospitals. Which strategy should the data scientist use to meet the business requirement?

Accepted Answer

Use SMOTE to oversample the minority class in the training set.. Using SMOTE (Synthetic Minority Over-sampling Technique) generates synthetic samples for the minority class, which can improve recall. Option A is wrong because collecting more data may not be feasible and may not help if imbalance persists. Option C is wrong because undersampling reduces data and may lose information. Option D is wrong because changing to a deep learning model may not help with limited data.

Answer

Collect more labeled data from other hospitals.

Answer

Use random undersampling of the majority class in the training set.

Answer

Switch to a deep neural network with more layers.