MLS-C01 Practice Test 5 — 25 Questions

Question 1

A company is building a sentiment analysis model for customer reviews. The dataset includes 10,000 positive and 10,000 negative reviews. The data scientist splits the data into 70% training, 15% validation, and 15% test sets. After training, the model achieves 99% accuracy on training set but only 82% on validation set. What is the most likely issue?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The model is overfitting. The 99% training accuracy versus 82% validation accuracy indicates the model has memorized the training data but fails to generalize to unseen data, which is classic overfitting. Option D is correct. Option A is incorrect because data leakage would typically cause both training and validation accuracy to be high and similar. Option B is incorrect because the dataset is balanced (10,000 positive and 10,000 negative). Option C is incorrect because underfitting would show low accuracy on both training and validation sets.

Answer

There is data leakage from validation to training

Answer

The dataset is imbalanced

Answer

The model is underfitting

Question 2

A company deploys a SageMaker endpoint for real-time inference. After a week, the response latency increases from 50 ms to 500 ms. CPU utilization is at 30%. What is the most likely cause?

Accepted Answer

The inference code makes a call to a downstream service that is throttling requests. Increased latency with low CPU utilization indicates that the model itself is not compute-bound. Instead, the bottleneck is likely external, such as a downstream service (e.g., database, API) that the inference code calls. Throttling by that service causes requests to queue up, increasing latency without raising CPU usage on the SageMaker instance. Option A would typically cause memory pressure rather than low CPU. Option B would show high CPU if underpowered. Option D would cause connection errors, not just latency increase.

Answer

The model has a memory leak

Answer

The instance type is underpowered for the inference load

Answer

The SageMaker endpoint is experiencing a network outage

Question 3

A company is using Amazon SageMaker to deploy a model for real-time inference. The model requires 500 MB of memory and has a latency requirement of 100 ms. The endpoint is receiving 10 requests per second. Which instance type should be chosen for cost-effectiveness?

Accepted Answer

ml.m5.large. Ml.m5.large (Option C). This instance type provides 2 vCPU and 8 GB memory, which is more than sufficient for the 500 MB memory requirement. It also offers consistent performance suitable for real-time inference with 10 requests per second and 100 ms latency. Option A (ml.c5.xlarge) has 4 vCPU and 8 GB, which is over-provisioned and more expensive. Option B (ml.t2.medium) has only 4 GB memory but uses burstable CPU, which may cause latency spikes. Option D (ml.p3.2xlarge) is GPU-optimized and significantly more expensive, making it unsuitable for a CPU-bound workload. Therefore, ml.m5.large is the most cost-effective choice.

Answer

ml.c5.xlarge

Answer

ml.t2.medium

Answer

ml.p3.2xlarge

Question 4

A machine learning engineer is evaluating a classification model that predicts whether a transaction is fraudulent. The model outputs a probability score. The cost of a false negative (missed fraud) is 10 times higher than the cost of a false positive (false alarm). Which TWO evaluation metrics should the engineer use to tune the model? (Choose TWO.)

Accepted Answer

F-beta score with beta = 2. Precision-Recall curve (E) is well-suited for imbalanced datasets and when the positive class (fraud) is rare; it directly evaluates the trade-off between precision and recall. F-beta score with beta = 2 (A) weights recall twice as much as precision, aligning with the higher cost of false negatives. Accuracy (B) is misleading for imbalanced data and does not incorporate costs. ROC-AUC (D) can be overly optimistic on imbalanced data and is less sensitive to the cost of false negatives. Log loss (C) measures probabilistic calibration but does not directly reflect the asymmetric cost structure.

Answer

Accuracy

Answer

Log loss

Answer

ROC-AUC

Question 5

A data scientist is training a binary classification model on an imbalanced dataset where the positive class represents 5% of the data. The model achieves 99% accuracy but only identifies 10% of the actual positive cases. Which metric should the data scientist focus on to evaluate the model's performance on the positive class?

Accepted Answer

Recall. Recall measures the proportion of actual positive cases that are correctly identified. In this imbalanced dataset, the model has high accuracy but low recall (only 10% of positives caught), so recall is the key metric to improve. Option A (Precision) is not the primary focus because it measures how many predicted positives are correct, not coverage. Option C (AUC-ROC) evaluates the model's ability to distinguish classes overall, not specifically the recall of the positive class. Option D (F1 score) is the harmonic mean of precision and recall, but since recall is very low, F1 is also low; however, recall directly addresses the problem of missing positives.

Answer

Precision

Answer

AUC-ROC

Answer

F1 score

Question 6

A machine learning engineer is using Amazon SageMaker to deploy a model for real-time inference. The model is a large ensemble that requires 4 GB of memory and has a latency requirement of 100 ms. Which instance type and deployment configuration should the engineer choose to optimize cost while meeting requirements?

Accepted Answer

ml.m5.large (2 vCPU, 8 GB memory). ml.m5.large provides 8 GB memory, sufficient for a 4 GB model plus overhead, and is cost-effective for real-time inference with moderate latency requirements. Option B (SageMaker Serverless Inference) is incorrect because cold start latency may exceed the 100 ms requirement. Option C (ml.c5.large) has only 4 GB memory, insufficient for the model. Option D (ml.p3.2xlarge) is GPU-accelerated and expensive, making it overkill for a non-GPU workload.

Answer

SageMaker Serverless Inference

Answer

ml.c5.large (2 vCPU, 4 GB memory)

Answer

ml.p3.2xlarge (8 vCPU, 61 GB memory, 1 GPU)

Question 7

A machine learning engineer is deploying a model using Amazon SageMaker. The model requires preprocessing steps (e.g., scaling, encoding) that were applied during training. Which TWO options can ensure the same preprocessing is applied at inference?

Accepted Answer

Create a Scikit-learn pipeline that includes preprocessing and the model, then deploy it.. Options D and E are correct. A Scikit-learn pipeline bundles preprocessing and the model into a single object, ensuring consistent preprocessing during training and inference. SageMaker Inference Pipeline chains a preprocessing container with the model container, allowing separate preprocessing steps to be applied consistently at inference time. Option A is wrong because using a Lambda function can introduce inconsistencies if not carefully managed, and it adds latency. Option B is wrong because a separate preprocessing endpoint adds complexity and may not guarantee identical preprocessing logic. Option C is wrong because retraining the model per inference request is impractical and computationally expensive.

Answer

Implement preprocessing as an AWS Lambda function invoked before inference.

Answer

Deploy a separate preprocessing endpoint and call it before the model endpoint.

Answer

Retrain the model in each inference request with the preprocessing applied.

Question 8

A company uses Amazon SageMaker to train a model using the built-in Linear Learner algorithm. The training data contains missing values in some features. What is the best practice for handling missing values with this algorithm?

Accepted Answer

Impute missing values using mean or median imputation. Linear Learner expects dense input; it cannot handle missing values. The best practice is to impute missing values before training, such as using mean or median imputation. Removing rows with missing values (Option A) may lose valuable data. Setting missing values to zero (Option C) could bias the model. The algorithm does not have a built-in `handle_missing` parameter (Option D). Therefore, Option B (Impute missing values using mean or median imputation) is correct.

Answer

Remove rows with missing values

Answer

Set missing values to zero

Answer

Use the `handle_missing` parameter in the algorithm

Question 9

A company uses Amazon SageMaker to train an XGBoost model on a large dataset. Training takes a long time. Which action can reduce training time without significantly affecting model accuracy?

Accepted Answer

Enable early stopping. Early stopping halts training when the model's performance on a validation set stops improving for a specified number of rounds. This prevents overfitting and reduces training time by eliminating unnecessary iterations, while typically preserving accuracy because the optimal model is already found.

Answer

Use a deep neural network instead

Answer

Increase the learning rate

Answer

Use a larger instance type

Question 10

Which TWO techniques can be used to detect multicollinearity among numerical features during exploratory data analysis? (Choose two.)

Accepted Answer

Compute a correlation matrix and look for pairs with absolute correlation > 0.8.. Multicollinearity indicates high correlation between predictors. Option B: Compute a correlation matrix and look for pairs with absolute correlation > 0.8 directly reveals linear dependencies. Option D: Variance Inflation Factor (VIF) measures how much the variance of a coefficient is inflated due to collinearity; VIF > 5 or 10 suggests multicollinearity. Option A (PCA) reduces dimensionality but does not directly detect collinearity. Option C (t-test) tests mean differences, not associations. Option E (chi-square) tests categorical independence, not applicable to numerical features.

Answer

Apply Principal Component Analysis (PCA) and examine loadings.

Answer

Perform a t-test between each pair of features.

Answer

Use a chi-square test of independence.

Question 11

A team is using SageMaker to train a deep learning model for image classification. The training job is failing with a 'CUDA out of memory' error. The team is using a p3.2xlarge instance (1 GPU, 16 GB GPU memory). The dataset consists of 256x256 RGB images. Which action is MOST likely to resolve the error without changing the instance type?

Accepted Answer

Reduce the batch size. The 'CUDA out of memory' error indicates that the GPU memory is exhausted. Reducing the batch size directly decreases the memory footprint per training step, allowing the model to fit within the 16 GB GPU memory of the p3.2xlarge instance. This is the most direct and effective fix without changing the instance type.

Answer

Increase the batch size to utilize GPU more efficiently

Answer

Enable automatic model tuning to optimize hyperparameters

Answer

Use Spot Instances to reduce cost

Question 12

A company uses a linear regression model to predict house prices. The model's R-squared is 0.95 on the training set but 0.60 on the test set. Which of the following is the most likely cause?

Accepted Answer

Overfitting. Such a large gap between training and test R-squared indicates overfitting. Option B is wrong because if the model were underfitting, both would be low. Option C is wrong because multicollinearity affects both training and test sets similarly and does not typically cause such a gap. Option D is wrong because data leakage would likely cause high test R-squared, not a significant drop.

Answer

Underfitting

Answer

Multicollinearity among features

Answer

Data leakage

Question 13

A data scientist is building a regression model to predict house prices. The dataset contains features like number of bedrooms, square footage, and location. After training, the model has high variance. Which technique should the data scientist use to reduce variance without significantly increasing bias?

Accepted Answer

Apply L2 regularization. L2 regularization (Ridge) penalizes large coefficients, effectively shrinking them and reducing model complexity, which lowers variance without substantially increasing bias. Option A (bagging) also reduces variance, but the question asks for a technique that reduces variance without significantly increasing bias; regularization is a more direct and common approach for high variance in regression. Option B (increasing features) would increase model complexity and variance, worsening the problem. Option D (fewer training examples) would increase bias and potentially variance due to less data.

Answer

Use bagging

Answer

Increase the number of features

Answer

Use fewer training examples

Question 14

A company is deploying a machine learning model using Amazon SageMaker. The model must be updated frequently without downtime. Which TWO strategies can achieve this? (Choose two.)

Accepted Answer

Use blue/green deployment with endpoint variants.. Amazon SageMaker supports blue/green deployment using endpoint variants, where you can deploy a new model version alongside the current one and then shift all traffic to the new variant once validated. This approach ensures zero downtime because the existing endpoint remains active during the transition, and traffic is switched atomically. Option E is correct because canary deployment with SageMaker allows you to gradually shift a small percentage of traffic to a new model variant, monitor its performance, and then ramp up to 100% if successful, all without interrupting the service.

Answer

Update the model artifact on the existing endpoint.

Answer

Delete the existing endpoint and create a new one.

Answer

Use rolling update with multiple instances.

Question 15

A company wants to deploy a machine learning model that requires very low latency predictions (under 10ms). The model is a small ensemble of decision trees. Which SageMaker deployment option is most suitable?

Accepted Answer

SageMaker endpoint with a single instance. C is correct because a SageMaker endpoint with a single instance provides a persistent, real-time inference API that can achieve sub-10ms latency for a small ensemble of decision trees. The endpoint keeps the model loaded in memory and uses synchronous HTTP requests, minimizing cold start and network overhead, which is essential for low-latency predictions.

Answer

SageMaker Notebook instance

Answer

AWS Lambda function with the model packaged

Answer

SageMaker Batch Transform

Question 16

A data scientist needs to run complex ETL transformations on a large dataset stored in Amazon S3. The transformations are written in PySpark and require occasional access to Hive metastore. The solution should minimize operational overhead and allow the data scientist to focus on code development. Which AWS service should be used?

Accepted Answer

Amazon EMR. Amazon EMR is the correct choice because it natively supports PySpark and Hive metastore integration, allowing the data scientist to run complex ETL transformations on large datasets stored in S3 with minimal operational overhead. EMR provides managed clusters that automatically scale and handle infrastructure, enabling the data scientist to focus on code development rather than cluster management.

Answer

Amazon Redshift

Answer

AWS Glue

Answer

Amazon SageMaker

Question 17

A data scientist trains a convolutional neural network (CNN) for image classification. The training loss decreases steadily, but the validation loss starts increasing after 10 epochs. Which technique should the data scientist use to address this problem?

Accepted Answer

Use early stopping to halt training when validation loss stops decreasing.. Early stopping halts training when validation loss stops improving, preventing overfitting. Option A (increasing data augmentation) may help reduce overfitting but does not directly address the already occurring validation loss increase. Option C (more epochs) would worsen overfitting. Option D (more convolutional layers) increases model capacity, likely worsening overfitting. Option E (higher learning rate) may cause divergence or instability.

Answer

Add more data augmentation to the training set.

Answer

Increase the number of training epochs.

Answer

Add more convolutional layers to increase model capacity.

Answer

Increase the learning rate.

Question 18

A data scientist is using Amazon SageMaker to train a model on a dataset that contains both numerical and categorical features. The categorical features have high cardinality (e.g., postal codes, product IDs). Which feature engineering approach is most suitable for handling these high-cardinality categorical features in a tree-based model?

Accepted Answer

Use label encoding. Label encoding is suitable for tree-based models because these models split on feature values and can handle ordinal relationships implicitly. Unlike linear models, tree-based models do not assume any distance metric between categories, so label encoding avoids the dimensionality explosion of one-hot encoding while preserving the ability to capture splits based on high-cardinality features.

Answer

One-hot encode the categorical features

Answer

Apply target encoding

Answer

Apply binary encoding

Question 19

A data scientist is using Amazon SageMaker to train a model using the built-in XGBoost algorithm. The training job is taking a long time. The data scientist notices that the input data is in CSV format and the training job is using File mode. The data size is 50 GB. What is the BEST way to reduce training time?

Accepted Answer

Switch the input mode to Pipe.. Pipe mode streams data directly from S3 to the training algorithm, reducing I/O overhead and improving throughput compared to File mode, which downloads the entire dataset first. Option A (larger instance) may not help if the bottleneck is I/O rather than compute. Option B (Parquet format) can improve performance but is not as impactful as Pipe mode for streaming. Option C (reduce features) could reduce training time but at the cost of model accuracy, making it not the best approach.

Answer

Use a larger instance type with more vCPUs.

Answer

Convert the data to Parquet format.

Answer

Reduce the number of features in the dataset.

Question 20

A data scientist is building a deep learning model using Amazon SageMaker. The model is overfitting the training data. Which THREE actions can help reduce overfitting?

Accepted Answer

Add L2 regularization to the loss function.. Overfitting can be reduced by regularization techniques such as L2 regularization (Option A) which penalizes large weights, by dropout (Option E) which randomly ignores neurons during training, and by data augmentation (Option B) which increases the effective size of the training dataset by creating modified copies. Increasing model complexity by adding layers (Option C) would worsen overfitting, and reducing the learning rate (Option D) does not directly address overfitting—it affects convergence speed.

Answer

Increase the number of layers in the network.

Answer

Reduce the learning rate.