MLS-C01 Practice Exam 4 — 20 Questions

Question 1

A machine learning team is building a model to predict customer churn. The dataset has 20 features and 50,000 rows. After initial EDA, they notice that the target variable 'churn' is highly imbalanced (5% churn, 95% non-churn). Which EDA step should the team prioritize to address this imbalance before model training?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Analyze the distribution of each feature separately for churn and non-churn groups.. During EDA for an imbalanced dataset, it is crucial to compare feature distributions between churn and non-churn groups to identify which features separate the classes. Option A is wrong because removing outliers from the majority class is not a standard EDA step and can introduce bias. Option C is wrong because stratified cross-validation is a model evaluation technique applied during training, not an EDA step. Option D is wrong because PCA is a dimensionality reduction technique that does not address class imbalance.

Answer

Remove outliers in the majority class to balance the dataset.

Answer

Perform stratified cross-validation to ensure balanced folds.

Answer

Apply Principal Component Analysis (PCA) to reduce noise.

Question 2

A data scientist uses Amazon SageMaker Data Wrangler to explore a dataset. The target column is 'price' (continuous). Which EDA analysis would best help decide between linear regression and tree-based models?

Accepted Answer

Check linear relationships between features and target. Checking linear relationships (e.g., scatter plots of features vs. target) helps determine whether linear regression is appropriate or if tree-based models (which capture non-linear patterns) would perform better. Option A (VIF) is used to detect multicollinearity, which affects linear regression but does not directly guide model selection between linear and tree models. Option C (Z-score) identifies outliers, which is important but not the primary factor for deciding between these model types. Option D (class imbalance) is relevant for classification problems, not regression.

Answer

Compute variance inflation factor (VIF) for features

Answer

Detect outliers using Z-score

Answer

Identify class imbalance in the target

Question 3

A data scientist is using Amazon SageMaker to train a linear regression model. The training data has 10 features and 100,000 observations. The model's training loss is decreasing, but the validation loss starts increasing after a few epochs. Which step should the data scientist take first to address this issue?

Accepted Answer

Reduce the learning rate. The increasing validation loss while training loss decreases is a classic sign of overfitting. Reducing the learning rate (Option B) is the first step to stabilize training by allowing the optimizer to take smaller, more controlled steps, which can help the model converge to a better local minimum and reduce validation loss. In SageMaker, this is typically adjusted via the `learning_rate` hyperparameter in the estimator.

Answer

Add more features to the model

Answer

Increase the batch size

Answer

Increase the number of epochs

Question 4

A data scientist is analyzing a dataset with missing values. The missing data mechanism is missing at random (MAR). Which imputation method is most appropriate to preserve relationships between variables?

Accepted Answer

Use multiple imputation by chained equations (MICE).. Multiple imputation by chained equations (MICE) is well-suited for missing at random (MAR) data as it models each variable with missing values conditional on other variables, preserving relationships. Option A (removing rows) reduces sample size and can introduce bias if data are not MCAR. Option B (KNN) assumes data are missing completely at random (MCAR) and may not handle MAR well. Option D (mean imputation) reduces variance and distorts relationships.

Answer

Remove all rows with any missing values.

Answer

Use k-nearest neighbors imputation.

Answer

Replace missing values with the mean of the column.

Question 5

A data scientist has this IAM policy attached to their IAM role. They are trying to run a SageMaker training job that reads data from 'my-bucket' and writes output to 'my-bucket'. The job fails. What is the most likely reason?

Accepted Answer

Missing iam:PassRole permission. The most likely reason the SageMaker training job fails is that the IAM role lacks the `iam:PassRole` permission. When SageMaker creates a training job, it must assume the execution role specified in the request; without `iam:PassRole`, the service cannot pass the role to itself, causing the API call to fail. This is a common prerequisite for any SageMaker job that requires an execution role.

Answer

The sagemaker:CreateTrainingJob action is not allowed on specific resources

Answer

Missing s3:ListBucket permission on the bucket

Answer

The training job requires permissions to write to CloudWatch Logs

Question 6

A company's ML model training on Amazon SageMaker is taking longer than expected. The training job uses a single ml.p3.2xlarge instance. Which change is most likely to reduce training time?

Accepted Answer

Use distributed training with multiple GPU instances. The training job is bottlenecked by compute capacity, as a single ml.p3.2xlarge instance provides only one NVIDIA V100 GPU. Distributed training with multiple GPU instances (e.g., multiple ml.p3.2xlarge instances) enables data parallelism, splitting the workload across GPUs and significantly reducing wall-clock training time for large models or datasets.

Answer

Increase the instance's EBS volume size

Answer

Enable Managed Spot Training

Answer

Switch to a compute-optimized instance with more vCPUs

Question 7

During EDA, a data scientist finds that a categorical feature 'city' has 500 unique values but only 10 cities account for 90% of the data. What is a recommended way to handle the rare categories?

Accepted Answer

Group rare categories into a single 'Other' category.. Grouping rare categories into 'Other' reduces cardinality, avoids overfitting from high-dimensional sparse features, and retains the majority of data from the top 10 cities. Option B (label encoding) is not recommended as it imposes an arbitrary ordinal relationship that may mislead the model. Option C (one-hot encoding all 500 categories) would create 499 dummy features, leading to the curse of dimensionality and sparse data. Option D (dropping rows with rare categories) discards potentially valuable data and may introduce bias.

Answer

Apply label encoding to all categories.

Answer

One-hot encode all 500 categories.

Answer

Drop all rows with rare categories.

Question 8

A data scientist is training a neural network for image classification. The dataset has 50,000 images across 100 classes. The model uses a ResNet-50 architecture pre-trained on ImageNet. The training loss decreases rapidly, but validation loss starts to increase after 5 epochs. Which of the following is the most effective technique to address this?

Accepted Answer

Use data augmentation to increase the diversity of the training set. The rapid decrease in training loss followed by an increase in validation loss after only 5 epochs is a classic sign of overfitting. Data augmentation artificially expands the training set by applying random transformations (e.g., rotations, flips, crops) to existing images, which improves the model's generalization and reduces overfitting. This is the most effective technique among the options because it directly addresses the lack of diverse training examples without changing the model architecture or training hyperparameters in a way that could destabilize learning.

Answer

Increase the learning rate

Answer

Add more layers to the network

Answer

Use a smaller batch size

Question 9

A team stores raw data in S3 and uses a Glue Data Catalog for metadata. They want to allow data scientists to query the data with Amazon Athena using their existing IAM roles. What is the MINIMUM set of permissions required?

Accepted Answer

Grant the IAM role permissions for Athena actions, Glue Data Catalog actions, and S3 read access.. Athena requires read access to S3 for querying data, permissions to the Glue Data Catalog for schema/metadata resolution, and Athena-specific actions to run queries. Write access to S3 is not required for querying, making B the minimum set.

Answer

Grant the IAM role permissions for Athena, Glue, and S3 (read and write).

Answer

Grant the IAM role permissions for Athena and Amazon Redshift Spectrum.

Answer

Grant the IAM role permissions for Athena and Amazon Kinesis.

Question 10

A data scientist is performing EDA on a dataset with mixed data types (numerical, categorical, text). The dataset is stored in S3. Which TWO AWS services can be used to directly perform statistical summaries and visualizations without writing custom code?

Accepted Answer

Amazon SageMaker Data Wrangler. Options D and E are correct. Amazon SageMaker Data Wrangler provides a visual interface for data preparation and analysis with built-in transforms and visualizations directly on S3 data. Amazon QuickSight is a BI service that connects to S3 and creates dashboards with statistical summaries and visualizations. Option A (SageMaker Studio) is an IDE for ML development, not a direct analysis service without custom code. Option B (AWS Glue DataBrew) is a data preparation tool but requires some configuration and is not primarily for statistical summaries and visualizations. Option C (Athena) is a SQL query engine for querying data, but does not provide built-in visualizations.

Answer

Amazon SageMaker Studio

Answer

AWS Glue DataBrew

Answer

Amazon Athena

Question 11

After loading a dataset into a pandas DataFrame, a data scientist runs df.info() and sees that a column 'income' has object dtype. What does this indicate, and what EDA step should be taken?

Accepted Answer

The column contains strings; convert to numeric using pd.to_numeric() and investigate non-convertible values.. 'object' dtype in pandas typically indicates string or mixed types. The appropriate EDA step is to attempt conversion to numeric using pd.to_numeric() and investigate non-convertible values to handle data quality issues. Option A is incorrect because object dtype does not specifically indicate missing values; missing values can appear in any dtype. Option C is premature; conversion should precede normalization. Option D is incorrect because object dtype is not numeric.

Answer

The column has missing values; impute them.

Answer

Normalize the column to a 0-1 range.

Answer

The column is already numeric; proceed.

Question 12

Refer to the exhibit. A data scientist runs the above AWS CLI command to create a SageMaker training job using the built-in Linear Learner algorithm. The training job fails with an error. What is the most likely cause?

Accepted Answer

The S3 data type is AugmentedManifestFile, but Linear Learner requires RecordIO or CSV. The command uses `S3DataType` as `AugmentedManifestFile`, but the Linear Learner algorithm only supports `RecordIO` or `CSV` as the S3 data type. AugmentedManifestFile is used for algorithms like object detection that require additional labels. The content type `application/x-recordio` is correct for RecordIO, but since the data type is set to AugmentedManifestFile, the training job fails. The IAM role, instance type, and MaxRuntimeInSeconds are all valid and would not cause this specific error. Therefore, the most likely cause is the incorrect S3 data type, which corresponds to option A.

Answer

The IAM role does not have sufficient permissions

Answer

The instance type ml.m5.large does not support the Linear Learner algorithm

Answer

The MaxRuntimeInSeconds is too short

Question 13

During EDA, a data scientist notices that a numeric feature 'age' has values ranging from 0 to 150, but expects adult ages between 18-100. Which TWO steps should the scientist take to investigate?

Accepted Answer

Compute summary statistics (min, max, percentiles). Computing summary statistics (min, max, percentiles) helps identify the range and potential outliers in the 'age' feature. Option E is correct because a box plot visualizes the distribution and clearly shows outliers, allowing the data scientist to investigate further. Option A is incorrect because removing rows with age > 100 without understanding the context may discard valid data (e.g., errors or special cases). Option C is incorrect because log transformation changes the scale but does not help in identifying outliers; it is used to handle skewed distributions. Option D is incorrect because imputing age values outside 18-100 with the mean would distort the distribution and is not appropriate for investigating outliers; it should only be considered after understanding the nature of the outliers.

Answer

Remove all rows with age > 100

Answer

Apply log transformation to normalize the distribution

Answer

Impute age values outside 18-100 with the mean

Question 14

A data scientist is training a model using Amazon SageMaker and wants to reduce the training time. The training job uses a single GPU instance. Which THREE actions can reduce training time?

Accepted Answer

Use distributed training across multiple GPU instances.. Options A, B, and C are correct. Distributed training across multiple GPU instances (A) leverages parallelism to reduce training time. Pipe input mode (B) streams data directly from S3, reducing I/O wait time compared to File mode which downloads data first. Using a larger instance type with more GPU memory and compute (C) provides more processing power, allowing faster training. Option D is incorrect because increasing the amount of training data typically increases training time, not reduces it. Option E is incorrect because reducing batch size can lead to more iterations and longer training time, though it may sometimes affect convergence.

Answer

Increase the amount of training data.

Answer

Reduce the batch size.

Question 15

A data scientist is performing EDA and observes that a feature 'purchase_amount' has many zeros and a long tail of positive values. What type of model would be appropriate for this target variable?

Accepted Answer

Zero-inflated negative binomial regression.. Zero-inflated negative binomial regression models are designed for count data with a high frequency of zeros, which matches the 'purchase_amount' feature having many zeros and a long tail of positive values. Option B is incorrect: Log transformation does not handle the zero-inflation problem; zeros become undefined or need adjustment. Option C is incorrect: Logistic regression is for binary outcomes, not continuous or count data. Option D is incorrect: Poisson regression accommodates count data but assumes the variance equals the mean and does not handle excess zeros; zero-inflation violates this assumption.

Answer

Linear regression after log transformation.

Answer

Logistic regression on binary indicator of purchase.

Answer

Poisson regression.

Question 16

A data scientist is working on a multi-class classification problem with 10 classes. The model outputs probabilities and the scientist wants to evaluate the model's ability to rank classes correctly. Which metric is most appropriate?

Accepted Answer

Area Under the ROC Curve (AUC-ROC). The most appropriate metric for evaluating a multi-class classifier's ability to rank classes (i.e., order classes by predicted probability) is the Area Under the ROC Curve (AUC-ROC). AUC-ROC measures the model's ability to distinguish between classes across all thresholds, and for multi-class problems it can be extended using one-vs-rest or macro/micro averaging. Log loss (Option D) measures probability calibration, not ranking. F1 score (Option A) is a threshold-dependent metric suitable for binary or per-class evaluation, not overall ranking. Accuracy (Option B) is also threshold-dependent and does not consider probability ranking. Therefore, Option C (AUC-ROC) is correct.

Answer

F1 score

Answer

Accuracy

Answer

Log loss

Question 17

A machine learning engineer is analyzing a dataset and observes that the distribution of a continuous feature is heavily right-skewed. Which transformation is most likely to make the distribution approximately normal?

Accepted Answer

Log transformation. A log transformation (C) is most appropriate for heavily right-skewed continuous data because it compresses the long right tail and can make the distribution approximately normal. Square root (A) is less effective for severe skewness. Exponential (B) would amplify the skewness. Box-Cox with lambda = 0 (D) is equivalent to log, but since log is explicitly given and commonly known, option C is the direct and correct choice.

Answer

Square root transformation

Answer

Exponential transformation

Answer

Box-Cox transformation with lambda = 0

Question 18

A data scientist is performing EDA on a dataset with a binary target variable. Which THREE techniques can help assess the relationship between a continuous feature and the target?

Accepted Answer

KDE plot grouped by target. Box plots (comparing distributions for each class), histograms (overlay or side-by-side), and KDE plots (probability density) are all effective for visualizing the relationship between a continuous feature and a binary target. Option A (scatter plot) requires two continuous variables. Option D (bar chart) is for categorical features.

Answer

Scatter plot against another continuous feature

Answer

Bar chart of feature values

Question 19

A data engineering team is designing a data pipeline that processes streaming data from Amazon Kinesis Data Streams using AWS Lambda. The team notices that some records are being processed multiple times (duplicates). Which TWO steps should the team take to ensure exactly-once processing?

Accepted Answer

Design the Lambda function to be idempotent.. Options A and B are correct. Making the Lambda function idempotent ensures that processing the same record multiple times does not cause duplicates downstream. Using a unique identifier per record and storing processed IDs in an external store like DynamoDB allows deduplication by checking if a record has already been processed. Option C is incorrect because increasing batch size does not prevent duplicates and may increase the chance of processing failures. Option D is incorrect because KPL provides exactly-once delivery to Kinesis Data Streams, not from the stream to Lambda, so deduplication is still needed. Option E is incorrect because disabling retries can lead to data loss without guaranteeing exactly-once processing.

Answer

Increase the batch size to reduce the number of invocations.

Answer

Use Kinesis Producer Library (KPL) to guarantee exactly-once delivery.

Answer

Disable retries on the Lambda function.

Question 20

Refer to the exhibit. A data scientist is unable to run an Amazon Athena query on data in `my-bucket`. The IAM policy shown is attached to the user. What is the most likely reason for the failure?

Accepted Answer

Athena needs s3:PutObject permission to write results.. Athena requires `s3:PutObject` permission to write query results to an output location. Without this permission, the query fails. Option A is not the issue because the bucket exists and is accessible; Option C is not necessarily relevant; Option D is incorrect because the bucket does exist (otherwise the user couldn't even attempt the query).

Answer

The ListBucket action is not granted.

Answer

The data is encrypted with SSE-C.

Answer

The bucket does not exist.