MLS-C01 Practice Test 15 — 15 Questions

Question 1

Which TWO techniques can help reduce overfitting in a decision tree model?

Accepted Answer

Limit the maximum depth of the tree. Limiting the maximum depth of the tree (Option C) directly restricts the number of splits, preventing the model from learning overly specific patterns in the training data. Pruning the tree after training (Option D) removes branches that have little predictive power, reducing variance and improving generalization. Both techniques combat overfitting by controlling the complexity of the decision tree.

Answer

Increase the number of trees in the forest

Answer

Increase the number of features considered per split

Answer

Increase the maximum depth of the tree

Question 2

A data scientist is training a linear regression model. After training, the model has a high bias and low variance. Which technique should the data scientist use to reduce bias?

Accepted Answer

Add more relevant features. High bias indicates the model is underfitting the data, meaning it is too simple to capture the underlying patterns. Adding more relevant features increases model complexity, allowing it to learn more from the data and reduce bias. This directly addresses the underfitting issue without increasing variance excessively, provided the features are meaningful.

Answer

Decrease the model complexity

Answer

Apply L2 regularization (Ridge)

Answer

Reduce the amount of training data

Question 3

A company is building a data lake on Amazon S3. Data arrives from multiple sources in different formats (CSV, JSON, Parquet). The engineering team wants to query this data using Amazon Athena with minimal transformation. Which approach minimizes query cost and improves performance?

Accepted Answer

Use AWS Glue to convert all data to Parquet format, partition by date, and store in a separate S3 bucket. Converting data to Parquet format (a columnar storage format) significantly reduces the amount of data scanned by Athena, which directly lowers query cost (Athena charges per TB scanned). Partitioning by date further limits scanned data by pruning irrelevant partitions. AWS Glue provides a serverless ETL service to perform this conversion efficiently, and storing the output in a separate S3 bucket avoids polluting the raw data lake.

Answer

Use Amazon Redshift Spectrum to query the data directly without transformation

Answer

Use Amazon EMR to convert data to CSV format and repartition

Answer

Store data as-is in S3 and create external tables in Athena for each format

Question 4

A company is using Amazon SageMaker to train a deep learning model on a large dataset stored in S3. The training job is failing with an OutOfMemory error. The data scientist wants to minimize cost while resolving the issue. Which action should the data scientist take?

Accepted Answer

Change the input mode from 'File' to 'Pipe'.. The OutOfMemory error occurs because the 'File' input mode downloads the entire training dataset to the instance's local storage before training begins, consuming significant memory. Switching to 'Pipe' mode streams data directly from S3 to the training algorithm, reducing memory footprint and avoiding the need for larger instances. This minimizes cost by using the existing instance type while resolving the memory issue.

Answer

Increase the instance type to one with more memory.

Answer

Use the 'auto' setting for the input mode.

Answer

Reduce the batch size hyperparameter.

Question 5

A data scientist uses Amazon SageMaker Data Wrangler to explore a dataset and notices that the target variable is highly imbalanced. Which technique should the data scientist apply to balance the dataset before training?

Accepted Answer

Synthetic Minority Oversampling Technique (SMOTE). Synthetic Minority Oversampling Technique (SMOTE) is the correct technique because it generates synthetic samples for the minority class by interpolating between existing minority instances and their k-nearest neighbors, effectively balancing the dataset without simply duplicating data. Amazon SageMaker Data Wrangler includes a built-in SMOTE transform, making it directly applicable for handling imbalanced target variables during exploratory data analysis.

Answer

One-hot encoding of the target variable

Answer

Random undersampling of the majority class

Answer

Min-Max scaling of all features

Question 6

Refer to the exhibit. A data scientist runs the AWS CLI command to create a SageMaker training job. The job fails immediately with 'ValidationException: Invalid instance type'. What is the most likely issue?

Accepted Answer

The training image URI is for a different AWS region. The error 'ValidationException: Invalid instance type' occurs because the training image URI specified in the command points to an Amazon ECR repository in a different AWS region than where the SageMaker training job is being created. SageMaker validates that the image URI is accessible from the current region; if the URI references a region that does not contain the XGBoost image or the instance type is not supported in that region's ECR, the validation fails. The instance type itself (ml.m5.large) is valid for XGBoost, but the mismatch between the image's region and the job's region triggers the exception.

Answer

The IAM role ARN is invalid

Answer

The S3 bucket 'my-bucket' does not exist or the role lacks permissions

Answer

The instance type ml.m5.large does not support the XGBoost image

Question 7

A company is using Amazon SageMaker to host a model for real-time inference. The model was trained using SageMaker's built-in Linear Learner algorithm. The endpoint has been running for a week, and the operations team notices that the endpoint's latency has increased from 50 ms to 150 ms over the past few days. The number of requests per second has remained steady at about 200. The team suspects a memory leak in the inference container. What should the team do to diagnose the issue?

Accepted Answer

Enable CloudWatch Logs and use Container Insights to view memory utilization.. CloudWatch Container Insights provides metrics for containerized applications, including memory utilization. This can help diagnose a memory leak by showing memory usage trends over time. Option B is incorrect because CloudWatch latency metrics only show endpoint response time, not memory usage. Option C is incorrect because SageMaker Debugger is designed for debugging training jobs, not inference containers. Option D is incorrect because SageMaker Model Monitor detects data and model drift, not memory leaks.

Answer

Use Amazon CloudWatch to monitor the endpoint's latency metric.

Answer

Use SageMaker Debugger to inspect the inference container.

Answer

Use SageMaker Model Monitor to detect data drift.

Question 8

A company uses SageMaker to deploy a model for predicting customer churn. The model was trained on historical data and achieves 85% accuracy on the test set. After deployment, the model's predictions are significantly worse on new data due to changes in customer behavior. What is the MOST likely cause?

Accepted Answer

Concept drift in the underlying data distribution. The model's performance degradation on new data, despite high accuracy on the test set, is a classic symptom of concept drift. Concept drift occurs when the statistical properties of the target variable (customer churn) change over time due to shifts in customer behavior, making the trained model's decision boundary obsolete. SageMaker deployed the model as a persistent endpoint, but the underlying data distribution has evolved, so the model no longer generalizes to the current environment.

Answer

Data leakage during training

Answer

The training dataset was too small

Answer

The model is overfitting to the training data

Question 9

A company uses SageMaker to host a real-time inference endpoint for a classification model. The endpoint receives traffic spikes that cause high latency. The team wants a solution that automatically scales based on demand while keeping costs low. Which approach is BEST?

Accepted Answer

Enable automatic scaling on the endpoint using Application Auto Scaling. SageMaker endpoints support automatic scaling with Application Auto Scaling based on custom metrics like 'InvocationsPerInstance' or 'SageMakerVariantInvocationsPerInstance'. Provisioned concurrency is not available for SageMaker endpoints. Spot instances are not recommended for real-time endpoints due to interruptions. Multi-model endpoints help but scaling is still needed.

Answer

Use provisioned concurrency for the endpoint

Answer

Use a multi-model endpoint to serve multiple models

Answer

Deploy the endpoint on Spot Instances

Question 10

A data scientist is evaluating a regression model. Which TWO metrics are appropriate for evaluating regression performance?

Accepted Answer

Root Mean Squared Error (RMSE). Root Mean Squared Error (RMSE) is a standard metric for regression models because it measures the average magnitude of prediction errors in the same units as the target variable. It penalizes larger errors more heavily due to squaring, making it sensitive to outliers, which is useful for evaluating model accuracy in continuous value prediction.

Answer

F1 score

Answer

Area Under the ROC Curve (AUC)

Answer

Precision

Question 11

A company is using Amazon Kinesis Data Analytics for Apache Flink to process real-time data. The data source is a Kinesis data stream, and the output is written to an S3 bucket. Recently, the processing latency has increased significantly. The team suspects that the Flink application is encountering backpressure. Which metric should the team monitor to confirm backpressure?

Accepted Answer

busyTimeMsPerSecond. The correct metric to confirm backpressure in a Flink application is `busyTimeMsPerSecond`. This metric measures the percentage of time a task is actively processing data versus waiting for input. A high `busyTimeMsPerSecond` value (close to 1000ms) indicates that the task is fully utilized and cannot keep up with the incoming data rate, which is the direct symptom of backpressure. Other metrics like `currentLowWatermark` relate to event time progress, not backpressure.

Answer

currentLowWatermark

Answer

numberOfFailedCheckpoints

Answer

numRecordsInPerSecond

Question 12

A company wants to serve a scikit-learn model via SageMaker. The inference code requires a custom preprocessing step that is not in the default scikit-learn container. What is the simplest way to deploy?

Accepted Answer

Create a custom Docker image extending the SageMaker scikit-learn container. Extending the SageMaker scikit-learn container with a custom Docker image is the simplest and most direct way to add custom preprocessing logic that is not included in the default container. SageMaker's pre-built scikit-learn container supports only standard scikit-learn inference code; any additional dependencies or custom preprocessing steps require you to build a custom image that inherits from the official SageMaker scikit-learn image and adds your code. This approach avoids the complexity of managing separate inference pipelines or external services.

Answer

Package the code in a Lambda layer and use SageMaker hosting

Answer

Use SageMaker Batch Transform with a custom processing script

Answer

Use SageMaker Neo to compile the model and add preprocessing

Question 13

A company is using Amazon SageMaker to deploy a real-time inference endpoint for a computer vision model. The endpoint receives bursts of traffic with up to 500 requests per second, but the load is unpredictable. Which scaling strategy is MOST cost-effective while maintaining low latency?

Accepted Answer

Configure automatic scaling with a target tracking policy and add a buffer to handle bursts. Amazon SageMaker's automatic scaling with a target tracking policy dynamically adjusts the number of instances based on a target metric (e.g., InvocationsPerInstance), which handles unpredictable bursts cost-effectively. Adding a buffer (e.g., a higher target value or a cooldown period) ensures low latency by pre-scaling before traffic spikes, avoiding cold starts and over-provisioning.

Answer

Manually provision enough instances to handle peak load

Answer

Use provisioned concurrency on SageMaker Serverless Inference

Answer

Use a multi-model endpoint to reduce the number of instances

Question 14

A company uses Amazon EMR with Spark to process data daily. The job reads from S3 and writes to S3. Recently, the job started failing with 'S3AccessDenied' errors. The IAM role used by EMR has not changed. What is the MOST likely cause?

Accepted Answer

The S3 bucket policy was updated to deny access to the EMR role. The IAM role used by EMR has not changed, but the S3 bucket policy can be updated independently to deny access to that specific role. An explicit deny in a bucket policy overrides any allow in the IAM policy, causing the 'S3AccessDenied' error even though the role itself remains unchanged. This is the most likely cause because the job was previously working and only the bucket policy could have been modified without touching the EMR configuration.

Answer

The EMR cluster's security group blocks outbound traffic

Answer

The S3 bucket was deleted and recreated

Answer

The EMR service role was rotated

Question 15

A data scientist needs to train a machine learning model using a large dataset (500 GB) stored in an S3 bucket. The training will be performed on a SageMaker notebook instance. The data scientist wants to minimize data transfer costs and reduce training time. Which data ingestion approach should the data engineer recommend?

Accepted Answer

Use the SageMaker SDK to directly read the data from S3 during training without copying it to the notebook.. The SageMaker SDK allows training jobs to read data directly from S3 using the Pipe or File mode, which avoids copying the 500 GB dataset to the notebook instance's EBS volume. This minimizes data transfer costs (no egress from S3 to the notebook) and reduces training time by streaming data directly to the training container without intermediate storage.

Answer

Copy the dataset to the notebook instance's attached EBS volume before training.

Answer

Load the dataset into an Amazon RDS database and query it from the notebook.

Answer

Mount the S3 bucket to the notebook instance using Amazon Elastic File System (EFS).