MLS-C01 Practice Test 43 — 15 Questions

Question 1

A data scientist is training a binary classification model on an imbalanced dataset where the positive class is rare. The model currently achieves 95% accuracy but only 10% recall on the positive class. Which metric should the data scientist prioritize to improve model performance?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

F1 score. The F1 score is the harmonic mean of precision and recall, making it the best single metric to optimize when the positive class is rare and both false positives and false negatives are costly. With 95% accuracy but only 10% recall, the model is likely predicting the majority class almost exclusively, so improving recall without sacrificing precision is critical — the F1 score directly balances this trade-off.

Answer

AUC-ROC

Answer

Precision

Answer

Accuracy

Question 2

A data engineer is designing a data lake on Amazon S3. The data is collected from IoT devices and is highly variable in volume. The engineer needs to ensure that the data is ingested reliably and can be processed in near real-time. Which AWS service should be used to ingest the data into the data lake?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the correct choice because it is a fully managed service designed to reliably load streaming data into data lakes on Amazon S3 with near-real-time latency (typically 60 seconds). It automatically handles scaling to accommodate highly variable IoT data volumes, provides built-in data transformation and compression, and requires no manual shard management or consumer code, making it ideal for ingestion into S3-based data lakes.

Answer

AWS Glue

Answer

Amazon Kinesis Data Streams

Answer

Amazon Simple Queue Service (SQS)

Question 3

A machine learning engineer is using Amazon SageMaker to train a model. The training job is taking too long. The engineer suspects the data loading is a bottleneck. Which action would MOST effectively diagnose the issue?

Accepted Answer

Use SageMaker Debugger to profile the training job. SageMaker Debugger provides built-in profiling that captures system metrics (CPU, GPU, memory, I/O) and framework metrics during training, and can identify data-loading bottlenecks via its profiling reports and rules. It is purpose-built to diagnose training performance issues, including input pipeline stalls. This makes it the most effective diagnostic action.

Answer

Monitor CPU utilization in CloudWatch

Answer

Enable SageMaker Model Monitor

Answer

Increase the instance type to a larger one

Question 4

A data scientist runs the AWS CLI command shown in the exhibit to list objects larger than 100 KB in an S3 bucket. The data scientist wants to understand the size distribution of these files. What is the most significant limitation of this approach for EDA?

Accepted Answer

The command may return incomplete results if there are more than 1000 objects.. The AWS CLI command likely uses the 'list-objects-v2' operation, which returns up to 1000 objects per request. If the bucket contains more than 1000 objects that match the criteria, the results will be incomplete unless pagination is used. This is the most significant limitation for EDA because it can lead to an inaccurate understanding of the size distribution.

Answer

The command only returns objects larger than 100 KB, not equal to.

Answer

The command uses the wrong query syntax and will fail.

Answer

The command does not return the file names, only sizes.

Question 5

A data scientist is training an LSTM model for time series forecasting using Amazon SageMaker. The model is overfitting. Which action is LEAST likely to reduce overfitting?

Accepted Answer

Increase the number of LSTM layers. Increasing the number of LSTM layers increases model capacity, which makes the network more expressive and therefore more prone to memorizing the training data — the opposite of what you want when overfitting. The other three options (dropout, fewer hidden units, early stopping) all reduce effective capacity or regularize training. In an overfitting scenario, adding depth without additional regularization typically worsens the train/validation gap.

Answer

Add dropout layers

Answer

Reduce the number of hidden units

Answer

Use early stopping

Question 6

A company uses SageMaker to train a model. The training job is taking too long and the data scientist wants to speed it up. Which THREE strategies should the data scientist consider? (Select THREE.)

Accepted Answer

Use a GPU instance type like ml.p3.2xlarge. Option B is correct because switching to a GPU instance type such as ml.p3.2xlarge provides hardware acceleration (NVIDIA V100 GPUs) that dramatically speeds up the matrix/tensor computations used in deep learning training compared to CPU-only instances. Option C is correct because distributed training spreads the workload across multiple instances (using SageMaker's distributed data parallel or model parallel libraries), reducing the wall-clock time of a training job by parallelizing gradient computation and updates. Option D is correct because Pipe input mode streams data directly from Amazon S3 to the training container instead of downloading the full dataset first, eliminating the initial download bottleneck and improving throughput for large datasets. Option A is not appropriate because reducing epochs changes the model's training convergence and can degrade accuracy rather than being a legitimate speed optimization. Option E is not appropriate because increasing batch size alone does not guarantee faster training and can affect convergence, memory usage, and model quality without addressing the underlying compute or I/O bottleneck.

Answer

Reduce the number of training epochs

Answer

Increase the batch size in the training script

Question 7

A data scientist is reviewing the training logs from a SageMaker training job. The model's loss decreases steadily and accuracy increases. However, when the model is evaluated on a holdout test set, the accuracy is only 0.65. Which issue does this behavior suggest?

Accepted Answer

The model is overfitting to the training data.. The model shows strong performance on the training set (loss decreases, accuracy increases) but poor generalization to the holdout test set (accuracy 0.65). This classic symptom indicates overfitting, where the model has memorized the training data rather than learning generalizable patterns. In SageMaker, this can occur when the model is too complex relative to the amount of training data, or when regularization techniques like dropout or L2 weight decay are insufficient.

Answer

The learning rate is too high.

Answer

The model is underfitting the training data.

Answer

The training data is corrupted.

Question 8

A data scientist is training a regression model on a dataset with 50 features. After training a linear regression model, the model achieves an R-squared of 0.85 on the training set but only 0.55 on the test set. Which technique is most likely to reduce the generalization error?

Accepted Answer

Apply L2 regularization (Ridge regression). The model exhibits high variance (overfitting): high training R² (0.85) but much lower test R² (0.55). L2 regularization (Ridge regression) shrinks coefficients toward zero, reducing model complexity and penalizing large weights, which directly combats overfitting and improves generalization to unseen data.

Answer

Add more features

Answer

Remove highly correlated features

Answer

Increase the polynomial degree of the model

Question 9

A company's ML pipeline uses AWS Step Functions to orchestrate data preprocessing, training, and evaluation. The training step occasionally fails due to a transient error. What is the most robust way to handle this without manual intervention?

Accepted Answer

Implement a retry policy with exponential backoff on the training step in the state machine. AWS Step Functions natively supports retry policies with exponential backoff, which automatically retries failed tasks after a delay that increases progressively. This handles transient errors (e.g., resource contention, network glitches) without manual intervention, making the pipeline robust and self-healing.

Answer

Configure a CloudWatch alarm to notify the team when the step fails

Answer

Use a parallel state to run multiple training instances simultaneously

Answer

Use a custom Lambda function to catch the error and restart the training step

Question 10

A data engineering team is building a real-time clickstream analytics pipeline on AWS. They need to ingest millions of events per second from mobile apps and websites, process them with low latency, and store the results in Amazon S3 for downstream analysis. Which combination of AWS services should the team use to minimize operational overhead while meeting these requirements?

Accepted Answer

Use Amazon Kinesis Data Streams for ingestion, Amazon Kinesis Data Analytics for real-time processing, and Amazon Kinesis Data Firehose to deliver results to Amazon S3.. Kinesis Data Streams handles high-throughput ingestion (millions of events per second), Kinesis Data Analytics (now Managed Service for Apache Flink) provides serverless real-time processing with SQL or Flink, and Kinesis Data Firehose delivers results to S3 without managing infrastructure. This combination is fully managed, scales automatically, and minimizes operational overhead while meeting low-latency and S3 delivery requirements.

Answer

Use Amazon MQ to ingest streaming data, AWS Lambda to process each message, and save output to Amazon S3.

Answer

Use Amazon Kinesis Data Streams to ingest data, Amazon EMR to process with Spark Streaming, and save output to Amazon S3.

Answer

Use AWS Glue to ingest data into Amazon RDS, then use AWS Glue ETL jobs to transform and load into Amazon S3.

Question 11

A data scientist needs to perform exploratory data analysis on a 100 GB CSV file stored in Amazon S3. The data is not sensitive. The scientist wants to use SQL queries to filter and aggregate the data without setting up a server or moving the data. Which service should be used?

Accepted Answer

Amazon Athena. Amazon Athena is the correct choice because it is a serverless, interactive query service that allows you to run standard SQL directly on data stored in Amazon S3 without any infrastructure setup. For a 100 GB CSV file, Athena can handle the query workload efficiently by automatically scaling, and it charges only for the data scanned per query, making it ideal for ad-hoc exploratory analysis without moving or transforming the data.

Answer

AWS Glue

Answer

Amazon EMR

Answer

Amazon Redshift Spectrum

Question 12

A company is building a binary classifier to predict equipment failure. The dataset has 99% negative (no failure) and 1% positive (failure) examples. The data scientist uses a random forest model with default settings. The model achieves 99% accuracy on the test set but fails to identify any actual failures. Which metric should the data scientist use to evaluate the model?

Accepted Answer

Recall. Recall (sensitivity) measures the proportion of actual positive cases correctly identified. With 99% negative examples, a model can achieve 99% accuracy by simply predicting 'no failure' for all instances, but this yields 0% recall for the failure class. Since the goal is to detect rare failures, recall is the appropriate metric to evaluate the model's ability to find positive cases.

Answer

RMSE

Answer

R-squared

Answer

Precision

Question 13

A data scientist is training a binary classification model on an imbalanced dataset where the positive class is rare. The model currently achieves 95% accuracy but only 10% recall on the positive class. Which metric should the data scientist prioritize to improve model performance?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Area under the Precision-Recall curve (PR AUC). In a highly imbalanced dataset where the positive class is only 1%, accuracy is misleading because a model can achieve 99% accuracy by simply predicting the negative class for all samples, resulting in 0% recall for the positive class. The Area under the Precision-Recall curve (PR AUC) is the correct metric because it focuses on the performance of the positive class by evaluating the trade-off between precision and recall, making it sensitive to changes in the minority class. Unlike ROC AUC, which can be overly optimistic in imbalanced settings due to the large number of true negatives, PR AUC provides a more realistic assessment of model performance for rare events.

Answer

Area under the ROC curve (ROC AUC)

Answer

Accuracy

Answer

F1 score

Question 14

Which THREE of the following are common issues that can be identified during exploratory data analysis? (Select THREE.)

Accepted Answer

Multicollinearity between features. Multicollinearity occurs when two or more features in a dataset are highly correlated, meaning they contain redundant information. During exploratory data analysis (EDA), correlation matrices and variance inflation factor (VIF) calculations can reveal this issue, which can destabilize linear regression models and inflate coefficient standard errors.

Answer

High latency in API endpoints

Answer

Gradient vanishing in neural networks

Question 15

A company is deploying a model that predicts customer churn. The model's recall for the churn class is 0.9, but precision is 0.4. The business cost of false positives is high. Which strategy would MOST likely improve precision without significantly harming recall?

Accepted Answer

Increase the decision threshold for the churn class. Adjusting the decision threshold to require a higher probability before predicting churn can reduce false positives (increase precision) but may lower recall. The goal is to find a threshold that balances both. Using more aggressive regularization or different algorithms may not directly control the trade-off.

Answer

Collect more data for the churn class

Answer

Use a different algorithm such as Random Forest

Answer

Decrease the decision threshold for the churn class