MLS-C01 Practice Test 21 — 15 Questions

Question 1

During EDA, a data scientist notices that a feature has a high proportion of missing values (e.g., 70%). The feature is continuous and expected to be important based on domain knowledge. What is the best approach to handle this?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Create a binary indicator for missingness and impute the continuous values with the median.. It preserves the predictive signal from the feature while accounting for the pattern of missingness. Creating a binary indicator allows the model to learn whether missingness itself is informative, and median imputation is robust to outliers for a continuous feature. This approach avoids the bias of dropping the feature entirely and is more principled than arbitrary out-of-range imputation.

Answer

Remove the feature entirely to avoid bias.

Answer

Impute missing values with -1 since it is out of range.

Answer

Drop all rows with missing values in that feature.

Question 2

Match each ML model evaluation concept to its definition.

Accepted Answer

Precision: The proportion of true positive predictions among all positive predictions. Precision, Recall, F1 Score, and ROC AUC are key evaluation metrics. Common confusions include swapping Accuracy, Specificity, and Precision definitions.

Answer

ROC AUC: The proportion of correct predictions among total predictions

Answer

Accuracy: The ability to correctly identify negative cases

Answer

Specificity: The proportion of true positive predictions among all positive predictions

Question 3

A data engineer is building a data pipeline using AWS Glue. The pipeline reads data from Amazon S3, transforms it, and writes it back to S3 in a different format. The engineer needs to handle schema evolution (new columns added over time). Which TWO features of AWS Glue can help manage schema evolution?

Accepted Answer

AWS Glue Data Catalog. Option A, the AWS Glue Data Catalog, is correct because it serves as the central metadata repository that stores table schemas, partition information, and schema versions, allowing the pipeline to detect and reconcile newly added columns as the underlying S3 data evolves. Option B, AWS Glue DynamicFrame, is correct because it is a schema-flexible abstraction that resolves schema inconsistencies at runtime, supports the ResolveChoice and relationalize transformations, and can automatically merge or reconcile new columns without requiring a rigid, pre-defined schema. Option C, AWS Lake Formation, is not correct because it is a permission and governance service for managing access to data lakes, not a mechanism for handling schema evolution. Option D, Amazon Athena, is not correct because it is an interactive query service that reads from the Data Catalog but does not itself manage or evolve schemas for Glue ETL jobs. Option E, Amazon S3 object tags, is not correct because tags are key-value metadata for classification, lifecycle, and access control, and they do not describe or evolve the columnar schema of the data.

Answer

AWS Lake Formation

Answer

Amazon Athena

Answer

Amazon S3 object tags

Question 4

Which TWO actions are appropriate when dealing with outliers in a dataset during exploratory data analysis? (Select TWO.)

Accepted Answer

Apply log transformation to reduce the impact of extreme values.. Applying a log transformation compresses the range of the data, reducing the influence of extreme values without removing them. This is a common technique in exploratory data analysis for right-skewed distributions, as it can make the data more normally distributed and improve the performance of models that assume normality.

Answer

Replace the mean with the median for numerical features.

Answer

Remove all outliers without further investigation.

Answer

Assume outliers are errors and delete them.

Question 5

A data scientist is evaluating a binary classification model that predicts whether a customer will churn. The model achieves an AUC of 0.85 on the test set. Which TWO statements about AUC are correct? (Choose two.)

Accepted Answer

AUC represents the probability that a randomly chosen positive instance is ranked higher than a randomly chosen negative instance.. Option A is correct because AUC (Area Under the ROC Curve) has a well-known probabilistic interpretation: it equals the probability that a randomly selected positive instance receives a higher predicted score than a randomly selected negative instance, which is exactly what this statement describes. Option E is correct because the ROC curve is constructed by plotting TPR against FPR at every possible classification threshold, so AUC aggregates ranking performance across all thresholds rather than depending on any single cutoff. Option B is wrong because an AUC of 0.5 corresponds to random guessing, while 0.85 indicates strong discriminative ability. Option C is wrong because average precision is the area under the precision-recall curve (AP/PR-AUC), a different metric that is more sensitive to class imbalance. Option D is wrong because AUC is threshold-independent and is not equivalent to accuracy at the 0.5 threshold, which depends on a specific decision cutoff.

Answer

An AUC of 0.85 indicates the model is no better than random guessing.

Answer

AUC is the average precision across all thresholds.

Answer

AUC is equivalent to the accuracy of the model at the default threshold of 0.5.

Question 6

A data scientist is training a binary classification model on a highly imbalanced dataset where the positive class represents only 1% of the data. The model achieves 99% accuracy but only identifies 5% of the actual positives. Which metric should the data scientist use to evaluate model performance?

Accepted Answer

Recall. Recall (sensitivity) measures the proportion of actual positives correctly identified by the model. With only 5% of positives detected, recall is 0.05, which directly reveals the model's failure to capture the minority class despite high accuracy. In imbalanced datasets, accuracy is misleading because the model can achieve 99% accuracy by simply predicting the majority class (negative) for all instances.

Answer

Mean squared error

Answer

Accuracy

Answer

Precision

Question 7

A machine learning engineer is deploying a model to an Amazon SageMaker endpoint. The model requires GPU for inference. Which instance type should be selected?

Accepted Answer

ml.p3.2xlarge. The ml.p3.2xlarge instance type is part of the Amazon SageMaker P3 family, which is specifically designed for GPU-accelerated computing. It includes NVIDIA Tesla V100 GPUs, making it suitable for deep learning inference that requires GPU. The other options are general-purpose (M5), compute-optimized (C5), and memory-optimized (R5) instances that do not have GPUs.

Answer

ml.m5.large

Answer

ml.c5.xlarge

Answer

ml.r5.large

Question 8

A company runs a real-time fraud detection system using Amazon SageMaker. The model is deployed as a SageMaker endpoint and receives predictions within milliseconds. Recently, the model's accuracy has degraded due to data drift. The data scientists want to monitor the model's performance continuously. What is the most effective way to detect data drift?

Accepted Answer

Enable Amazon SageMaker Model Monitor to capture inference data and compare it against a baseline dataset. Amazon SageMaker Model Monitor can capture inference data and compare it against a baseline dataset to detect data drift automatically. Option A is wrong because storing data in S3 and using Athena for SQL queries is a batch-oriented approach that lacks automated drift detection capabilities. Option B is wrong because CloudWatch anomaly detection on invocation count and latency metrics monitors operational health, not data drift. Option D is wrong because CloudWatch Logs Insights can query inference logs but does not provide automated drift detection.

Answer

Store all incoming requests in Amazon S3 and use Amazon Athena to run periodic SQL queries for drift detection

Answer

Set up Amazon CloudWatch anomaly detection on the endpoint's invocation count and latency metrics

Answer

Use Amazon CloudWatch Logs Insights to analyze inference logs and set custom alarms

Question 9

A data scientist is using SageMaker to train a random forest model. The dataset has 100 features and 1 million rows. The training job fails with a 'ResourceLimitExceeded' error. What is the MOST likely cause?

Accepted Answer

The account has reached its limit on the number of SageMaker training instances.. The 'ResourceLimitExceeded' error indicates that the account has reached its limit on the number of SageMaker training instances or vCPUs. Option A (S3 bucket region) would cause a different error, not a resource limit. Option B (GPU memory) is unlikely because random forest models typically use CPU instances. Option C (wrong algorithm) would result in an algorithm-specific error, not a resource limit. Option D correctly identifies that the account limit has been exceeded.

Answer

The S3 bucket containing the training data is not in the same region.

Answer

The instance type selected does not have enough GPU memory.

Answer

The wrong algorithm was specified for the training job.

Question 10

A company is streaming e-commerce events to Amazon Kinesis Data Streams. The data science team needs to join events from multiple shards in near real-time and then store the joined results in Amazon S3. Which solution would meet these requirements with the LEAST operational overhead?

Accepted Answer

Use Amazon Kinesis Data Analytics for Apache Flink to read from the Kinesis stream, perform a join operation using Flink SQL, and write the results to S3 using a sink connector.. Amazon Kinesis Data Analytics for Apache Flink is a fully managed service that can read from Kinesis Data Streams, perform stateful stream processing (including joins across shards), and write results to S3 using a built-in sink connector. It handles shard enumeration, checkpointing, and scaling automatically, minimizing operational overhead. This directly meets the requirements of near real-time join and S3 storage without managing infrastructure.

Answer

Use AWS Lambda functions with Kinesis triggers to process each record, join across shards using a DynamoDB table for state, and write to S3.

Answer

Use Amazon Kinesis Data Firehose to buffer the data and write to S3, then use Amazon Athena to join the data after it is stored.

Answer

Use AWS Glue ETL jobs that read from the Kinesis stream via the Kinesis connector and write the joined results to S3.

Question 11

A data scientist is training a binary classification model on a dataset with 100 features and 10,000 samples. The model achieves 99% accuracy on the training set but only 65% on the test set. Which technique should be applied first to address this issue?

Accepted Answer

Apply L2 regularization to the model. The 99% training accuracy versus 65% test accuracy is classic overfitting, and L2 regularization (ridge) is the standard first-line remedy because it penalizes large weights and constrains model complexity. With 100 features and only 10,000 samples, the model has ample capacity to memorize noise, so adding a weight-decay penalty directly addresses the variance problem. Other options either worsen overfitting or are irrelevant to the symptom.

Answer

Reduce the size of the training dataset

Answer

Increase the number of trees in a random forest

Answer

Add more features to the model

Question 12

A data scientist is analyzing a dataset with 100,000 observations and 50 features. The scientist uses a Jupyter notebook on Amazon SageMaker. During EDA, the scientist runs a command to check for missing values and notices that 20% of the data in one feature is missing. The missing values are not random; they are correlated with another feature. Which imputation method is MOST appropriate?

Accepted Answer

Multiple imputation by chained equations (MICE). MICE uses multiple imputation based on other features, accounting for the correlation between the missing feature and another feature. Option A is wrong because median imputation ignores the correlation and simply fills with the median, which does not leverage relationships between features. Option B is wrong because listwise deletion removes rows with missing data, which reduces sample size and can introduce bias if missingness is not completely random. Option C is wrong because mean imputation, like median imputation, ignores correlations and can distort relationships.

Answer

Median imputation

Answer

Listwise deletion (remove rows with missing values)

Answer

Mean imputation

Question 13

A team is building a regression model to predict house prices. They observe that the model performs well on training data but poorly on validation data. Which THREE actions can help reduce overfitting? (Choose THREE.)

Accepted Answer

Reduce model complexity by selecting fewer features. Option A is correct because reducing model complexity by selecting fewer features lowers the model's capacity to memorize noise in the training set, which directly addresses the high-variance problem causing the train-validation performance gap. Option B is correct because increasing regularization strength with L1 (Lasso) or L2 (Ridge) penalties adds a cost term to the loss function that shrinks coefficients, constraining the hypothesis space and discouraging overfitting. Option C is correct because collecting more training data gives the model more representative examples, reducing variance and making it harder to fit spurious patterns, which typically improves validation performance. Option D is incorrect because increasing the maximum depth of decision trees raises model capacity and makes overfitting worse, not better. Option E is incorrect because adding more interaction features expands the feature space and increases complexity, which also tends to worsen overfitting rather than reduce it.

Answer

Increase the maximum depth of decision trees

Answer

Add more interaction features

Question 14

An IAM policy attached to a SageMaker execution role is shown in the exhibit. When a data scientist tries to create a training job that writes logs to CloudWatch Logs, the job fails. What is the MOST likely reason?

Accepted Answer

The policy lacks permissions for CloudWatch Logs actions. The IAM policy attached to the SageMaker execution role must grant permissions for all actions the training job needs, including writing logs to CloudWatch Logs. If the policy lacks the necessary CloudWatch Logs actions (e.g., logs:CreateLogGroup, logs:CreateLogStream, logs:PutLogEvents), the job will fail when attempting to write logs. The other options are less likely because SageMaker API version is not specified in IAM policies, S3 bucket policy denial would affect S3 access not CloudWatch, and an implicit deny for SageMaker actions would prevent the job from starting at all.

Answer

The policy does not specify the SageMaker API version

Answer

The S3 bucket policy denies access to the training job

Answer

The policy has an implicit deny for SageMaker actions

Question 15

A company is using Amazon SageMaker to deploy a model for real-time inference. The endpoint receives variable traffic and the company wants to optimize cost while maintaining responsiveness. Which scaling policy should be used?

Accepted Answer

Target tracking scaling based on invocation count. Target tracking scaling based on invocation count is the correct choice because it automatically adjusts the number of instances in real-time based on a predefined metric (e.g., InvocationsPerInstance), maintaining responsiveness during variable traffic while optimizing cost by scaling down during low demand. This policy uses a target value (e.g., 1000 invocations per instance) and SageMaker Application Auto Scaling continuously monitors CloudWatch metrics to add or remove instances as needed, eliminating manual intervention.

Answer

Simple scaling with a cooldown period

Answer

Scheduled scaling

Answer

Manual scaling