MLS-C01 Mock Exam 78 — 25 Questions

Question 1

A data scientist is building a classification model and wants to evaluate its performance. Which TWO metrics are appropriate for a multi-class classification problem? (Choose 2)

Accepted Answer

Recall. Both precision and recall can be extended to multi-class via micro/macro averaging. R-squared is for regression; RMSE is for regression; Mean Absolute Error is for regression.

Answer

Mean Absolute Error (MAE)

Answer

R-squared

Answer

Root Mean Square Error (RMSE)

Question 2

A data scientist is working on a customer churn prediction project for a telecom company. The dataset contains 50,000 records with 25 features, including 'tenure' (number of months customer stayed), 'monthly_charges', 'total_charges', 'contract_type' (month-to-month, one year, two year), 'payment_method', and a target 'churn' (Yes/No). The data is stored in an S3 bucket as a single CSV file. The scientist uses Amazon SageMaker Data Wrangler to perform EDA. After importing the data, the scientist notices that the 'total_charges' column has many missing values (about 20% of rows). The scientist suspects that missing values occur only for customers with tenure = 0 (new customers). After verifying that suspicion, the scientist wants to handle the missing values appropriately. Which course of action should the scientist take?

Accepted Answer

Impute missing total_charges with 0, since missing values correspond to customers with tenure=0.. If total_charges is missing only for tenure=0, it means those customers have not been billed yet, so total_charges should be 0. Imputing with 0 is appropriate. Option A is wrong because dropping rows with missing total_charges would remove all new customers, biasing the dataset. Option B is wrong because imputing with mean would assign incorrect values to new customers. Option C is wrong because using a model to predict missing values is overkill and may introduce error when the true value is known to be 0.

Answer

Use a regression model to predict total_charges based on other features.

Answer

Impute missing total_charges with the mean of non-missing values.

Answer

Drop all rows with missing total_charges to avoid bias.

Question 3

A data scientist is performing exploratory data analysis on a dataset with mixed data types (numerical, categorical, text). The goal is to identify clusters of similar records. Which technique is most appropriate?

Accepted Answer

K-prototypes clustering. K-prototypes extends k-means to handle mixed data by combining Euclidean distance for numerical and Hamming distance for categorical. K-means only works with numerical data. DBSCAN works on numerical data. Hierarchical clustering typically uses numerical distance. Gower distance can be used but is less common in clustering algorithms.

Answer

DBSCAN

Answer

Hierarchical clustering

Answer

K-means clustering

Question 4

Which TWO services can be used to transform data in transit within a Kinesis Data Firehose delivery stream? (Choose 2)

Accepted Answer

AWS Lambda. AWS Lambda is correct because it can be invoked as a transformation function within a Kinesis Data Firehose delivery stream. When you enable data transformation, Firehose buffers incoming records and then calls a Lambda function you specify, passing batches of records for processing. The Lambda function can modify, enrich, filter, or reformat the data before Firehose continues delivering it to the destination.

Answer

Amazon Kinesis Data Analytics

Answer

Amazon Athena

Answer

Amazon S3

Question 5

A company operates a real-time fraud detection system using an Amazon SageMaker endpoint. The model is a gradient boosting model trained on historical transaction data. The endpoint is deployed on an ml.c5.2xlarge instance with auto-scaling enabled based on average latency. Recently, during a flash sale event, the endpoint started returning HTTP 503 errors. The CloudWatch metrics show that the CPU utilization is at 70%, and the average latency has increased from 50 ms to 200 ms. The auto-scaling policy is configured to add one instance when average latency exceeds 100 ms for 5 consecutive minutes, and remove one instance when latency drops below 50 ms for 5 minutes. The current number of instances is 2. The flash sale lasted 30 minutes. What should the company do to prevent this issue in future flash sales?

Accepted Answer

Pre-warm the endpoint by setting a minimum number of instances that can handle the expected peak load before the flash sale. The auto-scaling policy is reactive and requires a 5-minute evaluation period, which is too slow to handle the rapid traffic spike during a flash sale. Pre-warming the endpoint by setting a minimum number of instances to handle the expected peak load ensures capacity is available immediately. Option A (request throttling) would reject excess requests and cause errors, not prevent them. Option B (changing instance type) may help handle more load per instance but still suffers from the same reactive scaling delay, and it may be more expensive. Option D (simpler model) could reduce latency but may compromise model accuracy and does not address the scaling issue directly.

Answer

Enable request throttling to drop excess requests

Answer

Change the instance type to ml.c5.4xlarge to handle higher load

Answer

Change the model to a simpler model with lower latency

Question 6

A machine learning team is using Amazon SageMaker to train a model on a dataset stored in S3. The training job reads data from S3 using Pipe input mode, but the training is slow. The team wants to improve data throughput. Which THREE actions should they take?

Accepted Answer

Mount the S3 bucket using an S3 file system and use File mode with a larger instance type.. Mounting an S3 bucket using an S3 file system (e.g., via mount-s3 or s3fs) and switching to File mode allows the training instance to access data as local files, eliminating the overhead of streaming decompression and per-record parsing inherent in Pipe mode. Using a larger instance type provides more network bandwidth and CPU resources to handle the file I/O, directly improving data throughput for large datasets.

Answer

Enable S3 Transfer Acceleration on the bucket.

Answer

Use Amazon ElastiCache to cache the training data.

Question 7

A data scientist is training a binary classification model on a dataset with a severe class imbalance (95% negative, 5% positive). The model achieves 95% accuracy but only correctly identifies 10% of the positive class. Which metric should the data scientist use to evaluate model performance?

Accepted Answer

F1 score. The F1 score is the harmonic mean of precision and recall, making it robust to class imbalance. With 95% accuracy but only 10% recall on the positive class, the model is essentially a trivial classifier that predicts the majority class. F1 score captures both false positives and false negatives, providing a balanced view of performance on the minority class.

Answer

Log loss

Answer

Accuracy

Answer

Area under the ROC curve (AUC)

Question 8

A healthcare company is building a model to predict patient readmission within 30 days. They have structured electronic health records (EHR) data with 200 features. The data includes missing values, categorical variables with high cardinality (e.g., diagnosis codes), and a severe class imbalance (5% readmission). They need to deploy a model on SageMaker that is interpretable and achieves high recall for the positive class. Which combination of techniques should they use?

Accepted Answer

Use XGBoost with SMOTE, feature selection via SHAP, and deploy as a SageMaker endpoint. XGBoost natively handles missing values, making it well-suited for EHR data with missing entries. SMOTE addresses the severe class imbalance by generating synthetic samples of the minority class, which improves recall. SHAP feature selection provides interpretability by identifying the most influential features, and deploying as a SageMaker endpoint enables real-time predictions. This combination directly meets the requirements of high recall and interpretability.

Answer

Use logistic regression with one-hot encoding and random undersampling

Answer

Use PCA for dimensionality reduction, then train a linear SVM with class weights

Answer

Use a deep neural network with embeddings for categorical variables and oversample the minority class

Question 9

A data engineer is building a streaming pipeline using Amazon Kinesis Data Streams and AWS Lambda. The Lambda function processes records and writes results to Amazon S3. The engineer notices that the Lambda function is experiencing throttling and some records are being dropped. Which TWO actions should the engineer take to improve the reliability of the pipeline?

Accepted Answer

Increase the number of shards in the Kinesis data stream.. Increasing the number of shards in the Kinesis data stream directly increases the stream's throughput capacity. Each shard supports up to 1 MB/s write and 2 MB/s read, so more shards allow the stream to handle higher data volumes, reducing the likelihood of throttling and dropped records at the stream level.

Answer

Add a Dead Letter Queue to the Lambda function to capture failed records.

Answer

Decrease the batch size in the Lambda event source mapping.

Answer

Increase the Kinesis stream's retention period to 7 days.

Question 10

A company is building a data pipeline to process streaming data from IoT devices. The data is ingested via Amazon Kinesis Data Streams. Each record is about 1 KB. The company wants to use AWS Lambda for real-time transformations and then store the results in Amazon DynamoDB. The expected throughput is 10,000 records per second. The Lambda function currently runs in about 200 ms. The company is concerned about Lambda concurrency limits and wants to ensure there are no throttling errors. The default concurrency limit for Lambda is 1,000. Which approach should the team take to handle the expected throughput without throttling?

Accepted Answer

Increase the number of shards in the Kinesis Data Stream to 10 and set the Lambda batch size to 100.. Increasing the number of shards to 10 ensures that the Kinesis stream can support up to 10 concurrent Lambda invocations (one per shard). With a batch size of 100, each invocation processes 100 records, resulting in 100 invocations per second (10,000 records / 100 per batch). At 200 ms per invocation, the required concurrency is 100 * 0.2 = 20, well within the default 1,000 concurrency limit. Option A is incorrect because reducing the batch size to 10 would increase invocations to 1,000 per second, requiring 200 concurrent executions, which still fits but is less efficient; the main issue is that reducing batch size does not reduce throttling risk as it increases invocation rate. Option B is incorrect because Kinesis Data Firehose does not natively support Lambda for per-record transformations before writing to DynamoDB; it primarily targets S3, Redshift, or Elasticsearch. Option C is incorrect because increasing Lambda memory typically reduces execution time but does not lower concurrency requirements; moreover, 3,000 MB may not reduce time below 100 ms enough to avoid throttling at high throughput.

Answer

Increase the Lambda function memory to 3,000 MB to reduce the execution time below 100 ms.

Answer

Use Amazon Kinesis Data Firehose instead of Lambda to load data directly into DynamoDB.

Answer

Reduce the Lambda batch size to 10 so that each invocation processes fewer records, reducing the time per invocation.

Question 11

Which THREE steps should be taken to secure a SageMaker notebook instance that accesses sensitive data? (Select THREE.)

Accepted Answer

Enable encryption at rest for the notebook's EBS volume. SageMaker notebook instances use an Amazon EBS volume for storage, and enabling encryption at rest for this volume ensures that sensitive data stored on the notebook (e.g., datasets, model artifacts) is encrypted using AWS KMS-managed keys. This protects data at the storage layer, which is a fundamental security requirement for compliance with standards like HIPAA or PCI DSS.

Answer

Grant root access to the notebook instance for flexibility

Answer

Allow direct internet access from the notebook for downloading packages

Question 12

Refer to the exhibit. A data scientist is evaluating a binary classification model for spam detection. The exhibit shows a single prediction instance. What is the model's prediction for this instance?

Accepted Answer

Spam. The model's prediction is 'Spam' because the prediction instance shows a probability of 0.95 for the 'Spam' class, which exceeds the typical decision threshold of 0.5 used in binary classification. Since the probability for 'Spam' is higher than for 'Ham' (0.05), the model assigns the instance to the class with the highest probability, which is Spam.

Answer

Ham

Answer

0.95

Answer

The model is unsure because probability is not 1.0

Question 13

A data scientist is building a model to predict customer churn. The dataset contains categorical features with high cardinality (e.g., ZIP code, customer ID). Which encoding method is MOST suitable?

Accepted Answer

Target encoding. Target encoding is most suitable for high-cardinality categorical features because it replaces each category with the mean of the target variable for that category, effectively capturing the predictive signal while keeping the feature space dense. This avoids the curse of dimensionality from one-hot encoding and the arbitrary ordinality of label encoding, which can mislead tree-based models.

Answer

One-hot encoding

Answer

Label encoding

Answer

Hashing encoding

Question 14

A data scientist ran a hyperparameter tuning job for an XGBoost model. The tuning job completed, but the best validation RMSE is 2.34. The data scientist believes the model can perform better. Based on the exhibit, which change to the tuning strategy is most likely to improve the model's performance?

Accepted Answer

Increase the static num_round hyperparameter to 500. Increasing the static `num_round` hyperparameter to 500 allows the model to train for more boosting rounds, which can reduce underfitting and lower the RMSE further. The current best validation RMSE of 2.34 suggests the model may not have converged, and additional rounds can help the XGBoost model learn more complex patterns, provided overfitting is monitored with early stopping.

Answer

Use random search instead of Bayesian optimization

Answer

Change the objective to binary:logistic

Answer

Increase the maximum value of eta to 1.0

Question 15

Refer to the exhibit. A data scientist queries the table with 'SELECT COUNT(*) FROM mytable' in Athena and gets a result of 1000 rows. However, the scientist knows there are 1500 data files in the S3 location. What is the most likely reason for the discrepancy?

Accepted Answer

Some files may be empty or contain only headers, so they contribute 0 rows.. Athena counts rows from data files; if files are empty or contain only headers, they contribute 0 rows. With 1500 files and only 1000 rows, it is plausible that many files are empty or header-only, especially if the data pipeline produces such files. Option A is incorrect because Athena still parses lines even with a delimiter mismatch, treating each line as a row (though columns may be incorrect). Option B is incorrect because schema mismatch typically causes query errors, not silent skipping. Option D is false because Athena does not skip files based on size limits.

Answer

Some files may use a different delimiter (e.g., tab) and are not parsed correctly, resulting in zero rows from those files.

Answer

The table schema does not match the data, causing some files to be skipped.

Answer

Athena skips files larger than a certain size to prevent scanning too much data.

Question 16

A machine learning engineer needs to deploy a model that requires low latency (under 10 ms) for real-time inference. The model is a small ensemble of decision trees. Which Amazon SageMaker endpoint configuration is MOST appropriate?

Accepted Answer

Real-time endpoint. Real-time endpoints in Amazon SageMaker are designed for low-latency inference (typically under 10 ms) and are the correct choice for deploying a small ensemble of decision trees that needs to respond to individual prediction requests in real time. They keep the model loaded and ready, providing a persistent HTTPS endpoint that can serve predictions with minimal overhead.

Answer

Batch transform

Answer

Training job

Answer

Multi-model endpoint

Question 17

A company is deploying a machine learning model for real-time fraud detection. The model must have extremely low latency (<10 ms) and high throughput. Which THREE design choices meet these requirements? (Choose 3.)

Accepted Answer

Use GPU instances (e.g., ml.p3) for the endpoint.. GPU instances like ml.p3 provide massively parallel compute capability that accelerates matrix operations common in deep learning models, enabling inference latencies under 10 ms. For real-time fraud detection, the GPU's high throughput and low latency are essential for processing thousands of transactions per second without bottlenecks.

Answer

Use one endpoint per model to avoid interference.

Answer

Use SageMaker Batch Transform for real-time predictions.

Question 18

A data scientist is using Amazon SageMaker to train a model with a custom Docker container. The training job fails with an error: 'Container exited with code 137'. What is the most likely cause?

Accepted Answer

The training instance ran out of memory.. Exit code 137 (128+9) indicates the container was killed by the SIGKILL signal, which typically occurs when the Linux Out-Of-Memory (OOM) killer terminates a process that has exceeded its memory allocation. In Amazon SageMaker, training instances have finite memory, and if the training algorithm or data loading exceeds that limit, the OOM killer forcibly stops the container, resulting in exit code 137.

Answer

The training data was corrupted.

Answer

The training job exceeded the maximum runtime.

Answer

The Docker entrypoint script was not found.

Question 19

A machine learning team is building a recommendation system for an e-commerce platform. They have user-item interaction data (clicks, purchases). They need to choose an algorithm that can capture both user and item latent factors and handle missing data. Which algorithm should they use?

Accepted Answer

Matrix factorization. Matrix factorization is the correct choice because it decomposes the user-item interaction matrix into lower-dimensional latent factors for users and items, capturing underlying patterns in preferences. It naturally handles missing data by learning from observed interactions only, making it ideal for recommendation systems with sparse data.

Answer

Linear regression

Answer

Principal component analysis (PCA)

Answer

Convolutional neural network (CNN)

Question 20

A team is using Amazon SageMaker to train a model and wants to automatically stop training when the model stops improving to save costs. Which SageMaker feature should they use?

Accepted Answer

SageMaker Debugger. SageMaker Debugger provides built-in rules, such as `LossNotDecreasing`, that automatically monitor training metrics and can halt a training job when the model stops improving. This directly addresses the requirement to stop training when improvement plateaus, saving costs. While Managed Spot Training (C) reduces cost by using spot instances, it does not inherently provide automatic early stopping based on model performance; early stopping must be implemented separately. SageMaker Experiments (A) track and compare runs but do not stop training. Automatic Model Tuning (D) can apply early stopping to hyperparameter tuning jobs, but it is not a feature of a single training job.

Answer

SageMaker Experiments

Answer

SageMaker Managed Spot Training with early stopping

Answer

SageMaker Automatic Model Tuning