MLS-C01 Practice Test 22 — 15 Questions

Question 1

A data scientist is training a binary classification model on imbalanced data (95% negative, 5% positive). Which metric is most appropriate for evaluating model performance?

Accepted Answer

Area Under the ROC Curve (AUC-ROC). AUC-ROC is the most appropriate metric for imbalanced binary classification because it evaluates the model's ability to distinguish between positive and negative classes across all classification thresholds, without being biased by the 95% negative majority. It measures the trade-off between true positive rate and false positive rate, making it robust to class imbalance.

Answer

R-squared

Answer

Mean Squared Error (MSE)

Answer

Accuracy

Question 2

A company uses Amazon Kinesis Data Firehose to ingest streaming data and deliver it to an S3 bucket. The data is in JSON format with a timestamp field. The data science team wants to query the data using Athena with partitioning by year/month/day. How should the S3 data be organized?

Accepted Answer

Configure Firehose to use dynamic partitioning with custom prefix. Kinesis Data Firehose dynamic partitioning evaluates a JSON key (such as the timestamp field) and writes records into Hive-style prefixes like year=2024/month=01/day=15/, which Athena can query with partition pruning. This avoids post-processing and delivers partitioned data directly to S3. It is the native, serverless way to achieve the requirement.

Answer

Store data in a single prefix and use Athena's 'partition projection' feature

Answer

Use AWS Glue crawler to partition the data after delivery

Answer

Use Amazon EMR to partition the data after delivery

Question 3

A machine learning engineer is performing exploratory data analysis on a large dataset stored in Amazon S3 using AWS Glue. The dataset contains a mix of numeric and categorical features. The engineer wants to efficiently compute summary statistics (e.g., mean, median, standard deviation) for the numeric columns. Which AWS service or feature should the engineer use to achieve this with minimal setup?

Accepted Answer

Use AWS Glue DataBrew to profile the dataset.. AWS Glue DataBrew provides a visual interface to profile data and compute summary statistics without writing code. Option A is wrong because launching an Amazon EMR cluster requires setup and management, which is not minimal. Option C is wrong because Amazon Athena requires writing SQL queries and does not automatically compute summary statistics. Option D is wrong because Amazon SageMaker Data Wrangler is a good tool but requires more configuration than DataBrew for simple summary statistics.

Answer

Launch an Amazon EMR cluster and use Spark.

Answer

Use Amazon Athena to run SQL queries on the data.

Answer

Use Amazon SageMaker Data Wrangler.

Question 4

A data scientist is training a text classification model using Amazon SageMaker's BlazingText algorithm. The dataset consists of 1 million documents, each labeled with one of 10 categories. The model achieves 92% accuracy on a held-out test set. However, when deployed, the model performs poorly on documents containing slang and typos. What should the data scientist do to improve model robustness?

Accepted Answer

Augment the training data by introducing common slang replacements and typos.. Augmenting the training data with slang replacements and typos directly addresses the distribution shift between the training set and real-world documents. By exposing the model to these variations during training, it learns to generalize to noisy text, improving robustness. This is a data-centric approach that targets the root cause: the model has not seen such patterns before.

Answer

Remove all documents with slang or typos from the training set.

Answer

Increase the embedding dimension from 100 to 300.

Answer

Increase the number of training epochs.

Question 5

A data scientist is using SageMaker to train a model. The training data is stored in an S3 bucket in a different AWS account. What is required to allow SageMaker to access the data?

Accepted Answer

Configure the SageMaker execution role with a policy that grants cross-account access to the S3 bucket.. SageMaker uses an IAM execution role to access resources. To allow cross-account access to an S3 bucket, the SageMaker execution role must have an IAM policy that grants s3:GetObject and s3:ListBucket permissions for the bucket, and the S3 bucket policy must also grant cross-account access to that role. This is the standard AWS mechanism for cross-account resource access.

Answer

Set up VPC peering between the two accounts.

Answer

Create a SageMaker notebook instance in the same account as the S3 bucket.

Answer

Launch the training job from a SageMaker notebook in the account containing the S3 bucket.

Question 6

A machine learning engineer needs to deploy a TensorFlow model to a SageMaker endpoint. The model expects a specific input format. The engineer has the model artifacts stored in an S3 bucket. Which step is REQUIRED to deploy the model?

Accepted Answer

Create a SageMaker Model object using the TensorFlow serving image.. To deploy a TensorFlow model to a SageMaker endpoint, you must create a SageMaker Model object that references the model artifacts in S3 and specifies the appropriate TensorFlow Serving container image. This image handles loading the model and exposing a RESTful or gRPC inference endpoint. Without this step, SageMaker cannot associate the artifacts with a serving container to start the endpoint.

Answer

Register the model in SageMaker Model Registry.

Answer

Create a SageMaker training job to re-train the model.

Answer

Save the model as a SavedModel format.

Question 7

A data scientist is training a binary classification model on imbalanced data (95% negative, 5% positive). Which metric is most appropriate for evaluating model performance?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

F1 Score. The F1 Score is the harmonic mean of precision and recall, making it ideal for imbalanced datasets where accuracy is misleading. With 95% accuracy but only 10% recall, the model is simply predicting the majority class (negative) almost always, so F1 Score captures the trade-off between false positives and false negatives better than accuracy or AUC-ROC.

Answer

RMSE

Answer

Accuracy

Answer

AUC-ROC

Question 8

A data engineer is responsible for managing a data lake on Amazon S3. The data lake contains CSV files from various sources, totaling 10 TB. The engineer needs to make this data queryable using Amazon Athena. However, Athena queries are currently taking a long time and scanning large amounts of data. The engineer has noticed that the CSV files are not partitioned, and there are no indexes. The engineer wants to improve query performance and reduce costs. The data is accessed frequently for the last 30 days, but older data is rarely queried. The engineer also wants to minimize the amount of data scanned by Athena. What should the engineer do?

Accepted Answer

Convert the CSV files to Parquet format and partition the data by date.. Converting CSV to Parquet reduces data scanned because Parquet is columnar and compressed, and partitioning by date enables partition pruning so Athena only reads the last 30 days of frequently accessed data. Together these dramatically improve query performance and lower cost.

Answer

Convert the CSV files to JSON format and use Athena to query them.

Answer

Create indexes on the S3 objects using AWS Glue.

Answer

Convert the CSV files to ORC format and create a view in Athena.

Question 9

A company uses Amazon SageMaker to train a custom TensorFlow model for image classification. The training job runs on a single ml.p3.2xlarge instance. The dataset contains 500,000 images stored in S3. The training time is too long (over 24 hours). The data scientist wants to reduce training time without changing the model architecture. The dataset is already in TFRecord format. The training script uses the default TensorFlow data pipeline. Which change will MOST significantly reduce training time?

Accepted Answer

Use SageMaker's distributed data parallelism with multiple instances.. Using SageMaker's distributed data parallelism with multiple instances increases the number of GPUs and splits the training data across them, directly reducing the compute time. Option A is incorrect because simply increasing the number of data files does not reduce the computational workload, and Pipe mode primarily helps with streaming data but does not accelerate model training. Option C is incorrect because switching from File to Pipe mode improves data loading but does not address the core compute bottleneck. Option D is incorrect because optimizing the data pipeline with tf.data.Dataset.prefetch and cache can improve I/O efficiency, but the most significant performance gain comes from scaling out the training across multiple GPUs.

Answer

Use SageMaker Pipe mode and increase the number of data files.

Answer

Switch the input mode from File to Pipe.

Answer

Optimize the data pipeline using tf.data.Dataset.prefetch and cache.

Question 10

A data scientist is training a binary classification model on imbalanced data (95% negative, 5% positive). Which metric is most appropriate for evaluating model performance?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Recall. Recall measures the proportion of actual positives that were correctly identified (TP / (TP + FN)). With 95% negatives, a model that predicts 'negative' for every sample achieves 99% accuracy while catching zero positives — recall exposes this failure directly by returning 0.0. Accuracy is misleading on imbalanced data because the majority class dominates the metric, so recall (often paired with precision/F1 or AUC-PR) is the right lens.

Answer

Accuracy

Answer

RMSE

Answer

Precision

Question 11

A data scientist is performing EDA on a dataset with both numeric and categorical features. Which TWO techniques are appropriate for visualizing the relationship between a numeric feature and a binary categorical target?

Accepted Answer

Violin plot grouped by target. It shows the full distribution of the numeric feature for each of the two target classes side by side, revealing differences in shape, spread, and central tendency between the binary groups. Option D (Box plot grouped by target) is also correct because it compactly compares the numeric feature's median, quartiles, and outliers across the two categorical target levels, making group differences easy to assess. Both techniques directly visualize a numeric-vs-binary-categorical relationship by conditioning the numeric distribution on the target. Option A (Histogram) shows the distribution of a single numeric variable but does not inherently encode the binary target unless overlaid, so it is not the intended grouped comparison. Option B (Stacked bar chart) is suited to relationships among categorical variables, not a numeric feature versus a binary target. Option E (Scatter plot) requires two numeric axes and is not appropriate when one variable is categorical.

Answer

Histogram

Answer

Stacked bar chart

Answer

Scatter plot

Question 12

A team has trained a deep learning model on Amazon SageMaker using a custom Docker container. They want to deploy the model to a SageMaker endpoint for real-time inference. Which format should the model artifacts be in?

Accepted Answer

A single .tar.gz file containing the model files.. Amazon SageMaker requires model artifacts to be packaged as a single .tar.gz file when using a custom Docker container for real-time inference. This compressed archive must contain the model files (e.g., model.pth, model.h5) and any necessary inference code, as SageMaker extracts the archive to the /opt/ml/model directory during deployment. The .tar.gz format ensures consistent extraction and compatibility with SageMaker's inference pipeline.

Answer

A folder on S3 with the model files.

Answer

No format requirement; any file works.

Answer

A .zip file containing the model files.

Question 13

A machine learning team is using Amazon SageMaker to tune hyperparameters for a neural network. They have defined a hyperparameter tuning job with a random search strategy. The training time per job is very long. Which strategy can reduce the total tuning time?

Accepted Answer

Enable early stopping to terminate poorly performing jobs.. Enabling early stopping allows SageMaker to terminate training jobs that are unlikely to produce better results based on the objective metric, which directly reduces total tuning time by freeing up compute resources for more promising hyperparameter combinations. This is especially effective with random search, where many trials may converge slowly or plateau.

Answer

Use a larger instance type for each training job.

Answer

Switch to Bayesian optimization.

Answer

Increase the number of training jobs.

Question 14

A data scientist is using Amazon SageMaker to train a linear regression model. The training data contains outliers. Which THREE techniques can mitigate the impact of outliers?

Accepted Answer

Remove observations with outlier values from the dataset.. Option A is correct because removing observations with outlier values eliminates the extreme data points entirely, so they can no longer distort the fitted linear regression coefficients. Option D is correct because winsorization caps extreme feature values at a specified percentile (e.g., the 5th and 95th percentiles), reducing the leverage of outliers while retaining those observations. Option E is correct because a robust loss function such as Huber loss behaves quadratically for small residuals but linearly for large residuals, limiting the influence of outliers on the learned parameters. Option B is not appropriate because adding layers increases model capacity and would tend to fit outliers even more closely, worsening their impact on a linear regression task. Option C is not correct in this context because standardization only rescales features to mean zero and unit variance; it does not reduce the relative influence of extreme values, so outliers remain outliers after transformation.

Answer

Increase the number of layers in the model.

Answer

Standardize the features to have mean zero and unit variance.

Question 15

A machine learning engineer is tuning hyperparameters for a gradient boosting model using Amazon SageMaker Automatic Model Tuning. The objective metric is validation accuracy. After several tuning jobs, the best accuracy achieved is 0.85, but the engineer suspects the model is overfitting. Which hyperparameter adjustment is most likely to reduce overfitting?

Accepted Answer

Increase the regularization parameter (e.g., lambda or alpha). Increasing the regularization parameter (lambda for XGBoost, alpha for other implementations) penalizes model complexity, which reduces overfitting by discouraging the model from fitting noise in the training data. In SageMaker Automatic Model Tuning, this is a common adjustment when validation accuracy plateaus while training accuracy remains high.

Answer

Increase the maximum depth of trees

Answer

Increase the subsample ratio

Answer

Increase the learning rate