MLS-C01 Mock Exam 85 — 25 Questions

Question 1

A data scientist is training a random forest model on a dataset with 50 features. After training, the model achieves 98% accuracy on the training set but only 85% on the test set. Which technique is most appropriate to reduce the generalization error?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Reduce the maximum depth of each tree. The gap indicates overfitting. Random forest can overfit if trees are too deep or if the number of trees is too high. Reducing the maximum depth of trees limits model complexity and helps generalization. Increasing the number of trees typically reduces overfitting but can also increase computational cost; however, reducing depth is more direct. Feature selection or PCA might help but are less direct than controlling tree complexity.

Answer

Apply Principal Component Analysis (PCA) to reduce dimensionality

Answer

Add more training data

Answer

Increase the number of trees in the forest

Question 2

A machine learning team is using Amazon SageMaker to build a regression model. The target variable is heavily right-skewed with a long tail. Which data transformation should the team apply to the target variable before training?

Accepted Answer

Log transformation. A log transformation compresses the range of the target and makes the distribution more symmetric, improving model performance.

Answer

One-hot encoding

Answer

Min-max scaling

Answer

Standardization (z-score)

Question 3

A team is training a large deep learning model on Amazon SageMaker. The training job is taking too long and they want to reduce training time without changing the model architecture. Which action is MOST effective?

Accepted Answer

Use a GPU instance (e.g., p3.2xlarge) for training. Using a SageMaker managed training instance with GPU (e.g., p3.2xlarge) provides significant acceleration for deep learning models due to parallel processing.

Answer

Switch to a compute-optimized instance like c5.4xlarge

Answer

Increase the batch size and learning rate proportionally

Answer

Use SageMaker Automatic Model Tuning with hyperparameter optimization

Question 4

Which THREE of the following are best practices for training a deep learning model on Amazon SageMaker?

Accepted Answer

Use Pipe mode for large datasets to reduce I/O overhead. Profiling GPU utilization helps identify bottlenecks. Using Pipe mode for large datasets reduces I/O. Setting up automatic model tuning (hyperparameter optimization) is a best practice. Training on a single instance is not a best practice for large models. Debugger is for monitoring, not for training acceleration.

Answer

Use SageMaker Debugger to automatically fix training errors

Answer

Train on a single instance to avoid distributed training overhead

Question 5

A data scientist is training a binary classification model on a dataset with 100,000 positive samples and 1,000 negative samples. The model achieves 99% accuracy on the test set but a very low F1 score. What is the most likely cause?

Accepted Answer

The model predicts the majority class most of the time due to class imbalance. The accuracy is high because the model predicts the majority class (positive) most of the time, but the F1 score is low because it fails to identify the minority class (negative) correctly. This is a classic symptom of class imbalance where the model is biased toward the majority class.

Answer

The test set contains only positive samples

Answer

The model is overfitting due to too many features

Answer

The model is underfitting due to insufficient training

Question 6

A data scientist is evaluating a binary classification model. The model's AUC-ROC is 0.95. Which TWO statements are true?

Accepted Answer

The model has excellent discriminative ability. AUC-ROC measures the model's ability to distinguish between classes across all thresholds. A high AUC (close to 1) indicates good performance. AUC-ROC is threshold-independent. It does not directly indicate accuracy or calibration.

Answer

The model has no false positives

Answer

The model is well-calibrated

Answer

The model's accuracy is at least 95%

Question 7

A data scientist is training a linear regression model on a dataset with 10 numerical features. After training, the model's R-squared value is 0.99 on the training set but only 0.60 on the test set. Which TWO of the following are appropriate actions to reduce overfitting? (Choose TWO.)

Accepted Answer

Use a subset of the most important features. Regularization (L1 or L2) penalizes large coefficients and reduces overfitting. Reducing model complexity by using fewer features or simplifying the model also helps. Adding more features would increase complexity and overfitting. Increasing the number of epochs is not relevant for linear regression (which has a closed-form solution).

Answer

Normalize the features

Answer

Add more features to the model

Answer

Increase the number of training epochs

Question 8

A company uses Amazon SageMaker to build a text classification model using a pre-trained BERT model. The dataset contains 10,000 labeled documents. The model is overfitting: training accuracy is 99%, validation accuracy is 85%. Which TWO of the following are most likely to help reduce overfitting? (Choose TWO.)

Accepted Answer

Increase the dropout rate during fine-tuning. Increasing dropout during fine-tuning adds regularization. Decreasing the learning rate can help the model converge to a better solution and prevent overfitting to the training set. Increasing batch size can sometimes regularize but is not as effective as dropout. Adding more layers increases model capacity and overfitting. Using a larger pre-trained model also increases capacity.

Answer

Add more transformer layers to the model

Answer

Increase the batch size

Answer

Use a larger pre-trained BERT model

Question 9

A company is building a recommendation system using collaborative filtering on Amazon SageMaker. The dataset contains user-item interactions with a long-tail distribution: a few items have millions of interactions, while most items have very few. The model currently uses matrix factorization with ALS. The recall@20 metric is low for niche items. Which modification would most likely improve recall for long-tail items?

Accepted Answer

Use implicit feedback with confidence weighting to downweight popular items. Implicit feedback models can incorporate confidence weights that downweight popular items, helping the model focus on less frequent items. Adding explicit features would not directly address the long-tail. Increasing the number of factors might help but could also overfit. Regularization is already present; adjusting it might not target the issue specifically.

Answer

Increase the regularization parameter to prevent overfitting

Answer

Add explicit features like item category and user demographics

Answer

Increase the number of latent factors in the matrix

Question 10

A data scientist is building a recommendation system using collaborative filtering. The dataset contains user-item interactions in a sparse matrix. The model will be trained on Amazon SageMaker using the built-in Factorization Machines algorithm. Which data format should the scientist use for the training data?

Accepted Answer

RecordIO-protobuf format with sparse features. Amazon SageMaker's Factorization Machines algorithm expects input in the 'application/x-recordio-protobuf' format for sparse data, or in CSV format for dense data. For sparse data, Protobuf is recommended for performance.

Answer

CSV format with all features as columns

Answer

JSON format with nested arrays

Answer

Parquet format

Question 11

A data scientist is training a gradient boosting model using SageMaker. The model is overfitting to the training data. Which TWO actions can help reduce overfitting? (Choose 2)

Accepted Answer

Increase the minimum child weight. Increasing the learning rate actually worsens overfitting; increasing max_depth increases model complexity. Reducing max_depth and increasing min_child_weight both regularize the model.

Answer

Increase the number of boosting rounds

Answer

Increase the learning rate

Answer

Use a larger training dataset

Question 12

A data scientist is training a deep learning model for image classification on Amazon SageMaker. The dataset consists of 10,000 images of size 224x224 pixels. The training job uses a single ml.p3.2xlarge instance. The data scientist notices that the GPU utilization is very low (~20%) and the training is slow. Which change would most likely improve GPU utilization?

Accepted Answer

Increase the number of data loader workers to load data in parallel. Low GPU utilization often indicates that the data loading pipeline is bottlenecked. Increasing the number of data loader workers can improve data throughput to the GPU, keeping it busy.

Answer

Use gradient accumulation

Answer

Use a larger instance type with more GPUs

Answer

Increase the batch size

Question 13

Refer to the exhibit. A SageMaker training job failed with the error shown. What is the most likely cause of this error?

Accepted Answer

The input data contains missing or invalid values. The error indicates that the input data contains NaN or infinite values. This is a data quality issue. The algorithm expects clean numeric values. The algorithm itself is fine; the training script may have a bug but the error specifically points to input data.

Answer

The training algorithm is not compatible with the data type

Answer

The training instance type is not powerful enough

Answer

The training script has a syntax error

Question 14

A machine learning engineer needs to deploy a model that makes real-time predictions with latency under 100ms. The model is a small ensemble of decision trees. Which AWS service is MOST suitable?

Accepted Answer

Amazon SageMaker endpoint. Amazon SageMaker provides real-time endpoints with low latency for model inference, and can host the ensemble as a single endpoint.

Answer

Amazon EMR with Spark Streaming

Answer

AWS Glue

Answer

AWS Lambda with custom container

Question 15

A company is using Amazon SageMaker to deploy a machine learning model for real-time inference. The model was trained using XGBoost and achieves high accuracy. However, during deployment, the endpoint returns a 'ModelError' when receiving input data. The input is a CSV string. What is the most likely cause?

Accepted Answer

The input data format does not match the model's expected format (e.g., CSV vs JSON). The most common cause of ModelError during inference is that the input format does not match what the model expects. XGBoost models typically expect CSV without headers. The serializer setting in SageMaker must be configured correctly. If the model expects text/csv but the endpoint is configured as JSON, the error occurs. The other options are less likely: model weights are loaded correctly if the model deployed, and the instance type affects latency not errors.

Answer

The inference instance type is too small

Answer

The model is not properly loaded into memory

Answer

The model weights are corrupted during deployment

Question 16

A data scientist is training a linear regression model and wants to handle multicollinearity among features. Which TWO actions are appropriate?

Accepted Answer

Use Ridge regression (L2 regularization). Ridge regression (L2) adds a penalty that can reduce the impact of correlated features. Removing one of the correlated features directly addresses multicollinearity. Lasso (L1) may also help but is less effective for groups of correlated features. Scaling features does not remove collinearity. Adding interaction terms increases multicollinearity.

Answer

Add interaction terms between features

Answer

Use Lasso regression (L1 regularization)

Answer

Scale all features to have zero mean and unit variance

Question 17

A company uses Amazon SageMaker to train a classification model. The training job fails with an error indicating that the algorithm requires a GPU but the instance type does not have one. The scientist used the built-in XGBoost algorithm. What should the scientist do to resolve the issue?

Accepted Answer

Choose a CPU instance type for the training job. XGBoost does not require a GPU; it can run on CPU. The error may be due to using a GPU-only algorithm version or misconfiguration. The simplest solution is to choose a CPU instance type. Installing a GPU version is unnecessary. Changing algorithm is not needed. Using a larger CPU instance can help but is not required. Option A: Choose a CPU instance type is correct. Option B: Installing GPU version is not needed. Option C: Changing algorithm is unnecessary. Option D: Using a larger instance may not address the issue if the instance type is still GPU-only.

Answer

Install a GPU-enabled version of XGBoost in the training container

Answer

Change the algorithm to a deep learning algorithm

Answer

Use a larger GPU instance type

Question 18

A data scientist is training a text classification model using Amazon SageMaker's built-in BlazingText algorithm. The dataset contains 1 million documents. Which TWO hyperparameters are most important to tune for improving model accuracy?

Accepted Answer

Learning rate. Learning rate and number of epochs are critical hyperparameters for training neural networks like BlazingText. They control how quickly the model learns and how long it trains.

Answer

Batch size

Answer

Loss function

Answer

Type of optimizer

Question 19

Which TWO of the following are appropriate use cases for Amazon SageMaker built-in algorithms?

Accepted Answer

Classifying customer churn using tabular data. XGBoost is suitable for tabular classification. BlazingText is for text classification on word embeddings. Image classification using custom CNNs may use built-in but not necessarily. Time series forecasting is not a built-in algorithm (use DeepAR). Reinforcement learning is not a built-in algorithm.

Answer

Reinforcement learning using Q-learning

Answer

Image classification using a custom CNN architecture

Answer

Time series forecasting using ARIMA

Question 20

A company is building a sentiment analysis model using Amazon SageMaker BlazingText. The training data consists of 100,000 product reviews. The data scientist wants to use the Word2Vec algorithm to generate word embeddings. Which configuration is required to use the continuous bag-of-words (CBOW) architecture?

Accepted Answer

Set the mode parameter to 'cbow'.. In BlazingText, the 'mode' parameter controls the training objective. Setting 'mode' to 'cbow' enables the continuous bag-of-words architecture. 'skipgram' is for skip-gram. 'batch_skipgram' is for large-scale skip-gram. 'supervised' is for text classification.

Answer

Set the mode parameter to 'supervised'.

Answer

Set the mode parameter to 'batch_skipgram'.

Answer

Set the mode parameter to 'skipgram'.