MLS-C01 Mock Exam 92 — 25 Questions

Question 1

A data scientist is working on a regression problem to predict house prices. The dataset has 80 features, including categorical variables with high cardinality (e.g., zip code with 10,000 unique values). The target variable is log-transformed. The data scientist trains a linear regression model and obtains an R² of 0.45 on the test set. To improve performance, the data scientist considers: A) Applying one-hot encoding to all categorical features and using Ridge regression. B) Using target encoding for high-cardinality features and using a tree-based model like XGBoost. C) Removing all categorical features and using polynomial features for numerical features. D) Using principal component analysis (PCA) on all features before training a linear model. Which approach is MOST likely to improve the model's performance?

Accepted Answer

Target encoding + XGBoost. Target encoding efficiently handles high-cardinality features, and tree-based models like XGBoost can capture non-linear relationships and interactions, likely improving R². One-hot encoding would create too many features, causing sparsity. Removing categories loses information. PCA may discard important information.

Answer

Remove categorical features and use polynomial features

Answer

One-hot encoding + Ridge regression

Answer

PCA on all features before linear regression

Question 2

A data scientist is training a text classification model using Amazon SageMaker's BlazingText algorithm. The dataset consists of 1 million documents, each labeled with one of 10 categories. The model achieves 92% accuracy on a held-out test set. However, when deployed, the model performs poorly on documents containing slang and typos. What should the data scientist do to improve model robustness?

Accepted Answer

Augment the training data by introducing common slang replacements and typos.. Option A is correct. Data augmentation with noise helps the model generalize to variations. Option B is wrong because removing such documents reduces training data. Option C is wrong because a larger embedding dimension may not help with slang. Option D is wrong because increasing epochs may lead to overfitting.

Answer

Remove all documents with slang or typos from the training set.

Answer

Increase the embedding dimension from 100 to 300.

Answer

Increase the number of training epochs.

Question 3

A machine learning engineer is deploying a sentiment analysis model using Amazon SageMaker. The model is a BERT-based transformer that takes up to 512 tokens. The engineer notices that inference latency is high (over 500 ms per request) on a single ml.c5.xlarge instance. The application requires latency under 100 ms. The model has already been optimized using half-precision (FP16). Which action should the engineer take to reduce latency?

Accepted Answer

Use a GPU instance such as ml.g4dn.xlarge. Option B (use a GPU instance) accelerates inference for transformers. Option A (increase batch size) can help throughput but not latency for single requests. Option C (reduce max sequence length) may hurt accuracy. Option D (use SageMaker Neo) is for compilation but may not achieve sub-100ms.

Answer

Reduce the maximum sequence length to 128

Answer

Increase the batch size for inference requests

Answer

Use SageMaker Neo to compile the model for the target instance

Question 4

A startup is deploying a machine learning model for real-time recommendation on Amazon SageMaker. The model is a TensorFlow model (1 GB) and the endpoint uses a single ml.c5.2xlarge instance. The inference latency is currently 500 ms per request. The startup expects traffic to increase 10x in the next month. They want to maintain latency under 500 ms. What is the most cost-effective solution?

Accepted Answer

Set up auto-scaling for the endpoint based on average latency or request count. Option A is correct because auto-scaling adds instances only when needed, handling increased traffic while keeping latency low. Option B (larger instance) is more expensive and may not be needed. Option C (GPU) is overkill and costly. Option D (batch) is not real-time.

Answer

Use SageMaker Batch Transform to process requests in batches

Answer

Switch to a GPU instance type for faster inference

Answer

Upgrade to a larger CPU instance type, such as ml.c5.4xlarge

Question 5

A company uses Amazon SageMaker to train a time-series forecasting model using the built-in DeepAR algorithm. The training data consists of daily sales for 1000 products over 2 years. The model performs well on most products, but for a few products with intermittent demand (sporadic sales), the predictions are poor. Which action should the data scientist take to improve predictions for these products?

Accepted Answer

Create a separate forecasting model specifically for intermittent demand products, using a model designed for such patterns (e.g., Croston's method).. Option A is correct. Creating separate models for different demand patterns allows specialized treatment. Option B is wrong because the dataset is already long enough. Option C is wrong because using a linear model may underfit. Option D is wrong because increasing training data does not help with intermittent patterns.

Answer

Use a linear regression model for all products.

Answer

Increase the context length of the DeepAR model to capture longer history.

Answer

Add more training data by including additional product categories.

Question 6

A company is building a fraud detection model using a random forest classifier. The dataset is highly imbalanced with 99% legitimate transactions and 1% fraudulent. The model currently achieves 99% accuracy on the test set, but the fraud recall is only 10%. The business requires at least 80% recall for fraud. The data scientist has tried oversampling the minority class and adjusting class weights, but recall remains below 40%. The dataset contains millions of transactions with hundreds of features. Which approach should the data scientist try next to improve fraud recall?

Accepted Answer

Use a gradient boosting machine (e.g., XGBoost) with scale_pos_weight parameter. Option D (gradient boosting with scale_pos_weight) is effective for imbalance. Option A (logistic regression) likely underperforms. Option B (undersampling) loses too much data. Option C (PCA) reduces features but may lose signal.

Answer

Randomly undersample the majority class to a 50:50 ratio

Answer

Apply PCA to reduce dimensionality before training

Answer

Use a logistic regression model with L2 regularization

Question 7

A data scientist is training a binary classifier to detect network intrusions. The dataset has 1,000 features and 10 million samples, but only 0.1% are positive (intrusions). The scientist uses XGBoost with scale_pos_weight set to 100. The model achieves a recall of 0.90 and precision of 0.05 on the test set. The business requires precision of at least 0.50 while maintaining recall above 0.80. Which technique should the scientist apply?

Accepted Answer

Tune the decision threshold on validation data to maximize F1 score. Option B (post-training threshold tuning) adjusts the decision threshold to trade off precision and recall. Option A (increase scale_pos_weight) will further increase recall but decrease precision. Option C (undersample majority) can help but may reduce recall. Option D (use random forest) may not achieve required precision.

Answer

Switch to a random forest classifier with class weights

Answer

Randomly undersample the majority class to achieve 1:1 ratio

Answer

Increase scale_pos_weight to 500

Question 8

A data scientist is using Amazon SageMaker to train a model using the built-in XGBoost algorithm. The training job is taking a long time. The data scientist notices that the input data is in CSV format and the training job is using File mode. The data size is 50 GB. What is the BEST way to reduce training time?

Accepted Answer

Switch the input mode to Pipe.. Option B is correct. Pipe mode streams data directly from S3, reducing I/O overhead. Option A is wrong because increasing instance count may not help if I/O is bottleneck. Option C is wrong because converting to Parquet may not be supported directly by XGBoost. Option D is wrong because reducing data harms accuracy.

Answer

Use a larger instance type with more vCPUs.

Answer

Convert the data to Parquet format.

Answer

Reduce the number of features in the dataset.

Question 9

A company is using Amazon SageMaker to train a linear learner model for predicting customer lifetime value. The target variable is right-skewed with a long tail. The data scientist applies a log transformation to the target variable and trains the model. The model achieves a low root mean squared error (RMSE) on the log scale. However, when the predictions are exponentiated back to the original scale, the RMSE is much higher. Which step should the data scientist take to improve the model's performance on the original scale?

Accepted Answer

Use a loss function that models the original distribution, such as Poisson or Tweedie. Option B (use a loss function like Poisson or Tweedie) is appropriate for non-negative skewed targets. Option A (remove outliers) may lose data. Option C (use a different algorithm) may not address the issue. Option D (increase regularization) may not help.

Answer

Increase the regularization strength

Answer

Remove outliers from the training data

Answer

Use a deep learning model instead of linear learner

Question 10

A company uses Amazon SageMaker to host a model for real-time inference. The model is a large ensemble that takes 2 seconds to load into memory. To reduce cold start latency, the data scientist uses SageMaker's managed warm pools. However, they notice that during a sudden traffic spike, new instances still experience high latency. What is the BEST way to ensure consistently low latency for all requests?

Accepted Answer

Configure auto scaling based on the number of active invocations to maintain a buffer of warmed instances.. Option C is correct. Using auto scaling with a target metric keeps a buffer of warm instances. Option A is wrong because increasing instance size reduces per-request time but cold start still occurs. Option B is wrong because Serverless Inference has its own cold start issues. Option D is wrong because scaling down aggressively increases cold starts.

Answer

Use a larger instance type to reduce model loading time.

Answer

Reduce the number of instances to minimize cold start frequency.

Answer

Switch to SageMaker Serverless Inference.

Question 11

A data scientist is building a binary classification model to predict whether a customer will subscribe to a service. The dataset contains 20 features, including categorical variables with high cardinality (e.g., zip code with 10,000 unique values). The scientist uses a logistic regression model and obtains a training AUC of 0.85 and a test AUC of 0.60. The scientist suspects overfitting due to high cardinality features. Which approach should the scientist use to address this issue?

Accepted Answer

Apply target encoding with smoothing to the zip code feature. Option C (target encoding with smoothing) reduces cardinality while preserving predictive power. Option A (one-hot encoding) increases dimensionality drastically. Option B (label encoding) may introduce ordinality issues. Option D (remove zip code) may lose important information.

Answer

Apply label encoding to the zip code feature

Answer

Remove the zip code feature entirely

Answer

Apply one-hot encoding to the zip code feature

Question 12

A machine learning engineer is training a regression model to predict house prices using Amazon SageMaker. The dataset contains 10,000 samples and 50 numerical features. After training a linear regression model, the engineer notices that the training loss is low, but the validation loss is high. The engineer suspects overfitting. The dataset is already normalized. Which action should the engineer take to reduce overfitting?

Accepted Answer

Add L2 regularization (weight decay) to the loss function.. Option B (add L2 regularization) is correct because it penalizes large weights and reduces overfitting in linear models. Option A (decrease batch size) can introduce noise but doesn't directly regularize. Option C (increase learning rate) may cause divergence. Option D (reduce feature count by PCA) could help but may lose information; regularization is more direct.

Answer

Increase the learning rate to speed up convergence.

Answer

Reduce the number of features using PCA.

Answer

Decrease the mini-batch size during training.

Question 13

A financial services company is building a fraud detection model using Amazon SageMaker. The dataset has 10 million transactions, with 0.1% fraudulent. They train an XGBoost model with default hyperparameters. The model achieves 99.9% accuracy on the test set, but only catches 10% of actual fraud cases. The company wants to maximize the number of fraud cases caught while keeping the false positive rate below 5%. The data scientist has already tried adjusting the class weights and threshold, but the recall is still low. What should the data scientist do next?

Accepted Answer

Use a different algorithm such as a balanced random forest or SMOTE with XGBoost. Option C is correct because the model is underfitting the minority class; XGBoost with default settings may not handle extreme imbalance well. Using a specialized algorithm like balanced random forest or SMOTE can improve recall. Option A (more data) may not help if the new data is also imbalanced. Option B (PCA) reduces dimensionality but not imbalance. Option D (larger instance) does not improve model performance.

Answer

Collect more data, especially fraudulent transactions, to balance the dataset

Answer

Apply PCA to reduce the number of features and prevent overfitting

Answer

Use a larger instance type to train for more epochs

Question 14

A data scientist is using Amazon SageMaker Autopilot to automatically build a binary classification model. The dataset has 50 features and 100,000 rows. After the experiment completes, the best candidate model achieves an F1 score of 0.85 on the validation set. However, when deployed to a real-time endpoint, the model's F1 score drops to 0.72 on production data. The data distributions between training and production are similar. What is the MOST likely cause of the performance drop?

Accepted Answer

The Autopilot pipeline used features that are not available at inference time (data leakage).. Option C is correct. Data leakage during Autopilot's feature engineering can lead to overly optimistic validation scores. Option A is wrong because similar distributions suggest no drift. Option B is wrong because Autopilot handles missing values. Option D is wrong because inference instances typically don't affect accuracy.

Answer

Concept drift occurred between training and production.

Answer

The production data contains missing values that were not present in training.

Answer

The inference endpoint uses a different instance type than training.

Question 15

A data scientist is training a linear regression model using Amazon SageMaker's built-in Linear Learner algorithm. The dataset has 500 features and 1 million rows. After training, the model's training RMSE is 2.5 and validation RMSE is 2.6, which is acceptable. However, the scientist notices that many feature coefficients are very small but non-zero, and the model takes a long time to train. The scientist wants to reduce training time while maintaining similar accuracy. Which action should the scientist take?

Accepted Answer

Increase the L1 regularization strength. Option B (increase L1 regularization) will drive many coefficients to zero, reducing effective features and thus training time. Option A (increase L2) shrinks coefficients but doesn't zero them out, so less impact on speed. Option C (increase batch size) may speed training but could affect convergence. Option D (use a different algorithm) is unnecessary.

Answer

Increase the mini-batch size

Answer

Switch to a neural network model

Answer

Increase the L2 regularization strength

Question 16

A company is building a recommendation system using matrix factorization. The dataset has 1 million users and 100,000 items. The data scientist trains a model using SageMaker's Factorization Machines algorithm. The model achieves a root mean squared error (RMSE) of 0.95 on the test set. However, the business requires RMSE below 0.90. The data scientist has already tuned hyperparameters like number of factors and learning rate. Which additional step should the data scientist take to improve RMSE?

Accepted Answer

Add side features such as user demographics and item categories. Option D (add side features) provides more information to the model. Option A (more factors) may overfit. Option B (more iterations) may not help if converged. Option C (use linear regression) is less powerful.

Answer

Increase the number of training iterations

Answer

Increase the number of factors to 1000

Answer

Use a linear regression model instead

Question 17

A data scientist is using Amazon SageMaker to train a linear learner model for regression. After reviewing the training logs, the data scientist notices that the loss is not decreasing and remains high. The learning rate is set to 0.01. The data is normalized. What should the data scientist do to improve convergence?

Accepted Answer

Try different learning rates, such as 0.001 or 0.1.. Option B is correct. The learning rate may be too high causing oscillation or too low causing slow convergence. Adjusting it can help. Option A is wrong because more epochs may not help if the learning rate is inappropriate. Option C is wrong because the data is already normalized. Option D is wrong because reducing batch size increases noise but may not resolve convergence issues.

Answer

Normalize the data again.

Answer

Reduce the mini-batch size.

Answer

Increase the number of epochs.

Question 18

A research team is training a deep learning model for image classification using Amazon SageMaker. The model is a convolutional neural network (CNN) with 50 layers. The team uses a single ml.p3.2xlarge instance. After 10 hours of training, the model has not converged and the loss is decreasing very slowly. The team suspects vanishing gradients. They want to diagnose and fix the issue without significant code changes. Which action should they take?

Accepted Answer

Modify the architecture to include residual connections (skip connections). Option A (use residual connections) directly addresses vanishing gradients. Option B (increase learning rate) may cause divergence. Option C (add more layers) worsens the problem. Option D (use batch normalization) helps but residual connections are more targeted for vanishing gradients.

Answer

Add more convolutional layers to increase model capacity

Answer

Use batch normalization after each convolutional layer

Answer

Increase the learning rate by a factor of 10

Question 19

A company wants to use Amazon SageMaker to train a model, but the training data contains personally identifiable information (PII). The data scientist needs to ensure that the PII is not accessible during training. The data is stored in S3. What is the MOST secure approach?

Accepted Answer

Use SageMaker's built-in mechanisms to encrypt data at rest and in transit, and ensure the training container does not have direct S3 access.. Option D is correct. Using SageMaker with a SageMaker-managed KMS key encrypts data at rest and in transit, and the training container cannot access S3 directly. Option A is wrong because encryption alone does not prevent access. Option B is wrong because Redshift adds complexity. Option C is wrong because a VPC alone does not encrypt data.

Answer

Configure a VPC for the SageMaker notebook and training job.

Answer

Load the data into Amazon Redshift and use Redshift ML.

Answer

Use server-side encryption with S3-managed keys (SSE-S3).

Question 20

A data scientist is training a time series forecasting model using Amazon SageMaker's DeepAR algorithm. The dataset contains daily sales data for 10,000 products over 2 years. The scientist splits the data chronologically: training on the first 18 months, validation on the next 3 months, and test on the last 3 months. The model performs well on validation but poorly on test. The data scientist suspects the model is overfitting to the validation period. Which action should the scientist take to improve test performance?

Accepted Answer

Use time series cross-validation with an expanding window. Option D (use cross-validation respecting time order) such as expanding window. Option A (increase training data) may help but not specifically address overfitting to validation. Option B (add more features) may worsen overfitting. Option C (reduce context length) may lose long-term patterns.

Answer

Reduce the context length to 30 days

Answer

Add more exogenous features like holidays and promotions

Answer

Use the entire dataset for training and ignore validation