MLS-C01 Practice Exam 68 — 20 Questions

Question 1

A team is training a binary classifier and obtains a confusion matrix with 100 true positives, 10 false positives, 20 false negatives, and 200 true negatives. What is the precision of the model?

Accepted Answer

0.91. Precision is calculated as TP / (TP + FP). With 100 true positives and 10 false positives, precision = 100 / (100 + 10) = 100 / 110 ≈ 0.909, which rounds to 0.91. This metric measures how many of the positive predictions were actually correct.

Answer

0.87

Answer

0.94

Answer

0.83

Question 2

A data scientist is training a deep learning model for object detection. The training loss decreases rapidly in the first few epochs but then plateaus at a high value. The validation loss starts increasing after a few epochs. Which adjustment is MOST likely to improve generalization?

Accepted Answer

Implement early stopping with a patience parameter. The described behavior—training loss plateauing at a high value while validation loss increases—is a classic sign of overfitting. Early stopping with a patience parameter halts training when validation performance stops improving, preventing the model from memorizing noise and thus improving generalization. This directly addresses the overfitting without altering the model architecture or data distribution.

Answer

Add more convolutional layers

Answer

Use more aggressive data augmentation

Answer

Increase the learning rate

Question 3

A data scientist is deploying a machine learning model using SageMaker and wants to automate the retraining pipeline. The training data is updated daily in an S3 bucket. Which combination of AWS services should the data scientist use to trigger a new training job when new data arrives?

Accepted Answer

Amazon CloudWatch Events to invoke an AWS Lambda function that starts a SageMaker training job. Amazon S3 event notifications can be sent to Amazon CloudWatch Events (via Amazon EventBridge), which then triggers an AWS Lambda function. The Lambda function contains code to start a SageMaker training job using the boto3 SDK. This serverless architecture provides a fully automated, event-driven pipeline that responds immediately when new data arrives in the S3 bucket, without polling or manual intervention.

Answer

Amazon SQS queue to store S3 events and a cron job to poll and start training

Answer

Use SageMaker Pipelines with a schedule to check for new data every hour

Answer

Amazon S3 event notification to directly start a SageMaker training job

Answer

Amazon CloudWatch Events to run an AWS Step Functions state machine that starts a SageMaker training job

Question 4

A machine learning engineer is tuning hyperparameters for a gradient boosting model using Amazon SageMaker Automatic Model Tuning. The objective metric is validation accuracy. After several tuning jobs, the best accuracy achieved is 0.85, but the engineer suspects the model is overfitting. Which hyperparameter adjustment is most likely to reduce overfitting?

Accepted Answer

Increase the regularization parameter (e.g., lambda or alpha). Increasing the regularization parameter (e.g., lambda or alpha in XGBoost) penalizes model complexity and helps reduce overfitting, making option A correct. Option B is incorrect because increasing maximum depth increases model complexity, leading to overfitting. Option C is incorrect because increasing subsample ratio (using more data per tree) can increase overfitting, while decreasing it often reduces overfitting. Option D is incorrect because increasing learning rate makes the model learn faster, which can lead to overfitting.

Answer

Increase the maximum depth of trees

Answer

Increase the subsample ratio

Answer

Increase the learning rate

Question 5

A data science team is training a large deep learning model using Amazon SageMaker. The training job is taking a long time because the model has many layers and the dataset is large. The team wants to reduce training time by distributing the training across multiple GPUs on a single instance, as well as across multiple instances. Which TWO actions should the team take? (Choose two.)

Accepted Answer

Configure the training job to use SageMaker's model parallelism (SMP) library for pipeline or tensor parallelism.. The SageMaker model parallelism (SMP) library is specifically designed to split large deep learning models across multiple GPUs using pipeline or tensor parallelism. This allows the team to train models that are too large to fit on a single GPU and to reduce training time by parallelizing computation across devices within and across instances.

Answer

Use SageMaker's distributed data parallelism (SMDDP) library to shard the model across GPUs.

Answer

Use SageMaker's managed training with a single instance containing multiple GPUs and enable data parallelism.

Answer

Set the instance type to a single GPU instance and rely on automatic model parallelism.

Question 6

A company wants to monitor SageMaker endpoints for data drift. Which TWO services can be used together to detect and alert on drift?

Accepted Answer

SageMaker Model Monitor. SageMaker Model Monitor (option B) continuously monitors models for data and quality drift. Amazon CloudWatch Alarms (option D) can be set up on Model Monitor's metrics to trigger alerts when drift is detected. SageMaker Data Wrangler (option A) is for data preparation, not monitoring. AWS CodePipeline (option C) is for CI/CD. Amazon CloudWatch Logs (option E) is for log storage and analysis, not for alerting on drift.

Answer

SageMaker Data Wrangler

Answer

AWS CodePipeline

Answer

Amazon CloudWatch Logs

Question 7

A data engineering team needs to ingest streaming data from thousands of IoT devices into Amazon S3 for near-real-time analytics. The data arrives in bursts and must be processed with minimal latency. Which AWS service is most appropriate for the ingestion layer?

Accepted Answer

Amazon Kinesis Data Streams. Amazon Kinesis Data Streams is the most appropriate ingestion layer because it is designed for real-time, low-latency data ingestion from thousands of sources, such as IoT devices. It can handle bursty traffic by scaling shards dynamically and provides sub-second to second-level latency for data to be available for processing, which meets the minimal latency requirement for near-real-time analytics.

Answer

Amazon Kinesis Data Firehose

Answer

Amazon SQS

Answer

Amazon S3

Question 8

A media company uses SageMaker to deploy a real-time inference endpoint for content recommendation. The model is a PyTorch model that uses GPU. The endpoint is deployed with an ml.p3.2xlarge instance. Over time, the endpoint's latency increases significantly during peak hours. The company has enabled auto scaling based on CPU utilization. However, the latency spikes occur even when CPU utilization is low. The model is stateless and the inference code is efficient. What is the MOST likely cause of the latency spikes?

Accepted Answer

Auto scaling is configured based on CPU utilization, but the bottleneck is GPU utilization. The model runs on GPU, so the bottleneck is GPU utilization, not CPU. Auto scaling based on CPU utilization does not help when the GPU is saturated. The latency spikes during peak hours suggest that the GPU is overloaded, but auto scaling is not triggered because CPU utilization remains low.

Answer

The model uses stateful processing that accumulates requests

Answer

The inference container has a memory leak that causes gradual slowdown

Answer

The instance type is too small for the model

Question 9

A data scientist is building a regression model to predict house prices. The dataset has 10 features, and the model shows high variance with a low bias. Which technique should the data scientist use to reduce variance?

Accepted Answer

Apply L2 regularization to the model.. L2 regularization (Ridge regression) penalizes large coefficients by adding a squared magnitude term to the loss function, which shrinks the model's weights and reduces variance without substantially increasing bias. This directly addresses the high-variance, low-bias symptom, making the model less sensitive to fluctuations in the training data.

Answer

Increase the depth of decision trees in the ensemble.

Answer

Add more features to the model.

Answer

Reduce the amount of training data.

Question 10

During exploratory data analysis, a data scientist notices that the correlation matrix of features shows many pairs with absolute correlation > 0.95. The dataset includes both numerical and categorical variables. Which technique is most appropriate to reduce multicollinearity while preserving the most information?

Accepted Answer

Apply Principal Component Analysis (PCA) to the features.. PCA is the most appropriate technique because it transforms correlated features into orthogonal principal components, effectively handling multicollinearity while preserving variance. Option B (using only one-hot encoded categorical features) discards numerical features and may not address multicollinearity; Option C (L1 regularization) is a modeling technique, not for EDA; Option D (removing one feature per highly correlated pair) is ad-hoc and can lose information compared to PCA which captures variance in fewer dimensions.

Answer

Use only one-hot encoded categorical features.

Answer

Apply L1 regularization during model training.

Answer

Remove one feature from each highly correlated pair.

Question 11

A company uses Amazon S3 to store historical transaction data in CSV format. The data is partitioned by transaction_date. A data analyst runs Amazon Athena queries that frequently filter on customer_id and transaction_date. The queries are slow and expensive. The team needs to improve query performance and reduce cost. Which combination of actions should the team take? (Choose TWO.)

Accepted Answer

Convert the data from CSV to Parquet format.. Converting CSV to Parquet (option C) improves performance because Parquet is a columnar storage format that reduces the amount of data scanned by Athena, especially when queries only select a subset of columns. It also uses efficient compression, reducing storage and data scanned. Reorganizing the partition order (option D) to have customer_id first (the most frequently filtered column) improves partition pruning, reducing the amount of data read. Option A (S3 Select pushdown) is not fully supported by Athena; Athena already uses S3 Select for certain formats, but it doesn't guarantee significant improvement and may not be applicable. Option B (JSON) is worse than CSV because JSON is typically larger and not columnar. Option E (increasing workers) is not applicable as Athena is serverless and automatically scales.

Answer

Enable S3 Select pushdown in Athena to reduce data transfer.

Answer

Convert the data to JSON format for better query performance.

Answer

Increase the number of Athena query workers.

Question 12

A company is using Amazon SageMaker Ground Truth to build a training dataset for an image classification model. The company has a large number of unlabeled images stored in Amazon S3. The data science team wants to use a private workforce consisting of internal employees to label the images. The team creates a labeling job with a private workforce. After starting the job, the team notices that the labeling tasks are not being assigned to any workers. The workers have been added to the private workforce and have received their login credentials. What is the MOST likely cause of this issue?

Accepted Answer

The workers have not been added to the work team that is assigned to the labeling job.. For a private workforce, workers must be added to a work team that is associated with the labeling job. If workers are not added to the work team, the labeling tasks will not be assigned to them. Option D correctly identifies that the workers have not been added to the work team. Option A (different task type) would affect the labeling interface, not task assignment. Option B (S3 bucket permissions) would prevent workers from viewing images but not from receiving tasks. Option C (IAM permissions) would prevent login, not task assignment.

Answer

The labeling job is configured for a different task type than image classification.

Answer

The S3 bucket containing the images has incorrect permissions, preventing workers from viewing the images.

Answer

The workers do not have the required IAM permissions to access the labeling portal.

Question 13

A data scientist is training a linear regression model and notices that the model performs well on training data but poorly on validation data. Which technique should be applied to reduce overfitting?

Accepted Answer

Apply L2 regularization (Ridge). L2 regularization (Ridge) adds a penalty term proportional to the square of the magnitude of the coefficients to the loss function. This shrinks the weights toward zero, reducing the model's sensitivity to individual features and preventing it from fitting noise in the training data, which directly addresses overfitting.

Answer

Increase the number of epochs

Answer

Add more features

Answer

Remove training examples

Question 14

A data scientist is using SageMaker to train a model using the built-in XGBoost algorithm. The training job fails with the error 'AlgorithmError: Framework error: No module named 'xgboost''. What is the most likely cause?

Accepted Answer

The training job is using a custom container that does not have XGBoost installed.. The built-in XGBoost algorithm requires the 'xgboost' Python package; SageMaker's built-in algorithms provide the necessary environment, but if the container is overridden or the wrong image is used, the module may be missing. Option A is wrong because the error is about missing module, not data format. Option C is wrong because the error is not about permissions. Option D is wrong because the error is not about output path.

Answer

The training data is not in CSV format.

Answer

The IAM role does not have permission to access SageMaker.

Answer

The S3 output path is incorrect.

Question 15

A data analyst is examining a dataset with a target variable that has three classes: A, B, C. They plot the distribution of a feature 'X' for each class and notice that for classes A and B, the distributions are bimodal, while for class C it is unimodal. They want to assess whether feature 'X' is useful for separating the classes. Which of the following metrics should they compute to quantify the separability?

Accepted Answer

ANOVA F-statistic between feature X and the target.. The ANOVA F-statistic measures the ratio of between-group variance to within-group variance, directly quantifying separability. Option B is wrong because 'variance ratio' is not the standard name; the correct metric is the F-statistic. Option C is wrong because the chi-square test is for categorical features, not continuous ones like feature X. Option D is wrong because mutual information measures dependency but does not specifically test separability in terms of variance between groups.

Answer

Variance ratio (between-group variance / within-group variance).

Answer

Chi-square test of independence.

Answer

Mutual information between X and the target.

Question 16

A data scientist is using SageMaker Debugger to monitor a training job. The training loss is not decreasing as expected. Which Debugger feature can help identify the issue?

Accepted Answer

Built-in rules to detect training anomalies. SageMaker Debugger's built-in rules are designed to automatically monitor training jobs for common issues such as vanishing gradients, overfitting, and loss not decreasing. When the training loss plateaus or fails to decrease, a rule like 'LossNotDecreasing' can trigger a CloudWatch alarm or stop the training job, providing immediate insight into the problem without manual inspection of tensors.

Answer

Automatic hyperparameter tuning

Answer

Saving tensors every step

Answer

Deploying a model endpoint for real-time monitoring

Question 17

A company uses Amazon SageMaker to train a linear regression model on a dataset with 10 million rows and 50 features. The training job takes 8 hours to complete. A data scientist wants to reduce the training time to under 2 hours without changing the dataset size or the model algorithm. The SageMaker instance type currently used is ml.m5.2xlarge. Which action should the data scientist take to achieve the desired training time?

Accepted Answer

Use SageMaker's distributed training with 4 ml.m5.2xlarge instances.. Using SageMaker's distributed training with multiple instances parallelizes the computation across 4 ml.m5.2xlarge instances, reducing wall-clock time by approximately a factor of 4, which can bring the 8-hour job down to around 2 hours. Option A (GPU instance) is not optimal for linear regression, which is CPU-bound. Option B (doubling vCPUs) provides only a 2x improvement, insufficient to reach under 2 hours. Option C (reducing features) changes the dataset and is not allowed per the requirement.

Answer

Change the instance type to ml.p3.2xlarge (GPU instance).

Answer

Change the instance type to ml.m5.4xlarge (double the vCPUs and memory).

Answer

Reduce the number of features from 50 to 25.

Question 18

A data scientist is performing exploratory data analysis on a dataset containing customer transactions. The dataset has 1 million rows with 50 features, including numerical and categorical variables. The goal is to identify patterns and potential data quality issues before building a model. Which approach should the data scientist take to efficiently explore the data?

Accepted Answer

Use AWS Glue DataBrew to profile the dataset, view data quality reports, and visualize distributions.. AWS Glue DataBrew is purpose-built for visual data preparation and profiling without writing code. It can directly profile the 1-million-row dataset, automatically generate data quality reports (e.g., missing values, outliers, data types), and provide distribution visualizations for both numerical and categorical features, making it the most efficient choice for exploratory data analysis.

Answer

Use Amazon Athena to run SQL queries and generate summary statistics.

Answer

Use Amazon SageMaker Data Wrangler to import the data and create a flow for feature engineering.

Answer

Use Amazon SageMaker Ground Truth to label the data and then analyze the labels.

Question 19

A company is using SageMaker built-in object detection algorithm to detect defects in manufacturing images. The model is trained on 10,000 labeled images and achieves 95% accuracy. However, in production, the model misclassifies many defective items as non-defective (false negatives). The business requires recall > 90% for the defect class. Which action should they take?

Accepted Answer

Adjust the decision threshold of the model to increase recall at the expense of precision. Threshold tuning directly optimizes recall for a given class.

Answer

Use a different algorithm such as semantic segmentation

Answer

Use SageMaker's Automatic Model Tuning to find better hyperparameters

Answer

Retrain the model with more images of non-defective items

Question 20

A research lab is training a large language model (LLM) on SageMaker using PyTorch. The model has 1 billion parameters and does not fit on a single GPU. They have access to a cluster of 16 p4d.24xlarge instances (each with 8 A100 GPUs). They need to train the model with minimal changes to the training script. Which SageMaker feature should they use?

Accepted Answer

SageMaker's model parallelism with automatic partitioning. SageMaker's model parallelism is designed for large models that don't fit on a single device.

Answer

SageMaker's distributed data parallelism with Horovod

Answer

Use SageMaker's built-in BlazingText algorithm

Answer

SageMaker's managed spot training with checkpointing