MLS-C01 Practice Test 7 — 10 Questions

Question 1

A machine learning team is building a model to predict customer churn. The dataset has 20 features and 50,000 rows. After initial EDA, they notice that the target variable 'churn' is highly imbalanced (5% churn, 95% non-churn). Which EDA step should the team prioritize to address this imbalance before model training?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Analyze the distribution of each feature separately for churn and non-churn groups.. During EDA for an imbalanced dataset, it is crucial to compare feature distributions between churn and non-churn groups to identify which features separate the classes. Option A is wrong because removing outliers from the majority class is not a standard EDA step and can introduce bias. Option C is wrong because stratified cross-validation is a model evaluation technique applied during training, not an EDA step. Option D is wrong because PCA is a dimensionality reduction technique that does not address class imbalance.

Answer

Remove outliers in the majority class to balance the dataset.

Answer

Perform stratified cross-validation to ensure balanced folds.

Answer

Apply Principal Component Analysis (PCA) to reduce noise.

Question 2

A data scientist uses Amazon SageMaker Data Wrangler to explore a dataset. The target column is 'price' (continuous). Which EDA analysis would best help decide between linear regression and tree-based models?

Accepted Answer

Check linear relationships between features and target. Checking linear relationships (e.g., scatter plots of features vs. target) helps determine whether linear regression is appropriate or if tree-based models (which capture non-linear patterns) would perform better. Option A (VIF) is used to detect multicollinearity, which affects linear regression but does not directly guide model selection between linear and tree models. Option C (Z-score) identifies outliers, which is important but not the primary factor for deciding between these model types. Option D (class imbalance) is relevant for classification problems, not regression.

Answer

Compute variance inflation factor (VIF) for features

Answer

Detect outliers using Z-score

Answer

Identify class imbalance in the target

Question 3

A data scientist is using Amazon SageMaker to train a linear regression model. The training data has 10 features and 100,000 observations. The model's training loss is decreasing, but the validation loss starts increasing after a few epochs. Which step should the data scientist take first to address this issue?

Accepted Answer

Reduce the learning rate. The increasing validation loss while training loss decreases is a classic sign of overfitting. Reducing the learning rate (Option B) is the first step to stabilize training by allowing the optimizer to take smaller, more controlled steps, which can help the model converge to a better local minimum and reduce validation loss. In SageMaker, this is typically adjusted via the `learning_rate` hyperparameter in the estimator.

Answer

Add more features to the model

Answer

Increase the batch size

Answer

Increase the number of epochs

Question 4

A data scientist is analyzing a dataset with missing values. The missing data mechanism is missing at random (MAR). Which imputation method is most appropriate to preserve relationships between variables?

Accepted Answer

Use multiple imputation by chained equations (MICE).. Multiple imputation by chained equations (MICE) is well-suited for missing at random (MAR) data as it models each variable with missing values conditional on other variables, preserving relationships. Option A (removing rows) reduces sample size and can introduce bias if data are not MCAR. Option B (KNN) assumes data are missing completely at random (MCAR) and may not handle MAR well. Option D (mean imputation) reduces variance and distorts relationships.

Answer

Remove all rows with any missing values.

Answer

Use k-nearest neighbors imputation.

Answer

Replace missing values with the mean of the column.

Question 5

A data scientist has this IAM policy attached to their IAM role. They are trying to run a SageMaker training job that reads data from 'my-bucket' and writes output to 'my-bucket'. The job fails. What is the most likely reason?

Accepted Answer

Missing iam:PassRole permission. The most likely reason the SageMaker training job fails is that the IAM role lacks the `iam:PassRole` permission. When SageMaker creates a training job, it must assume the execution role specified in the request; without `iam:PassRole`, the service cannot pass the role to itself, causing the API call to fail. This is a common prerequisite for any SageMaker job that requires an execution role.

Answer

The sagemaker:CreateTrainingJob action is not allowed on specific resources

Answer

Missing s3:ListBucket permission on the bucket

Answer

The training job requires permissions to write to CloudWatch Logs

Question 6

A company's ML model training on Amazon SageMaker is taking longer than expected. The training job uses a single ml.p3.2xlarge instance. Which change is most likely to reduce training time?

Accepted Answer

Use distributed training with multiple GPU instances. The training job is bottlenecked by compute capacity, as a single ml.p3.2xlarge instance provides only one NVIDIA V100 GPU. Distributed training with multiple GPU instances (e.g., multiple ml.p3.2xlarge instances) enables data parallelism, splitting the workload across GPUs and significantly reducing wall-clock training time for large models or datasets.

Answer

Increase the instance's EBS volume size

Answer

Enable Managed Spot Training

Answer

Switch to a compute-optimized instance with more vCPUs

Question 7

During EDA, a data scientist finds that a categorical feature 'city' has 500 unique values but only 10 cities account for 90% of the data. What is a recommended way to handle the rare categories?

Accepted Answer

Group rare categories into a single 'Other' category.. Grouping rare categories into 'Other' reduces cardinality, avoids overfitting from high-dimensional sparse features, and retains the majority of data from the top 10 cities. Option B (label encoding) is not recommended as it imposes an arbitrary ordinal relationship that may mislead the model. Option C (one-hot encoding all 500 categories) would create 499 dummy features, leading to the curse of dimensionality and sparse data. Option D (dropping rows with rare categories) discards potentially valuable data and may introduce bias.

Answer

Apply label encoding to all categories.

Answer

One-hot encode all 500 categories.

Answer

Drop all rows with rare categories.

Question 8

A data scientist is training a neural network for image classification. The dataset has 50,000 images across 100 classes. The model uses a ResNet-50 architecture pre-trained on ImageNet. The training loss decreases rapidly, but validation loss starts to increase after 5 epochs. Which of the following is the most effective technique to address this?

Accepted Answer

Use data augmentation to increase the diversity of the training set. The rapid decrease in training loss followed by an increase in validation loss after only 5 epochs is a classic sign of overfitting. Data augmentation artificially expands the training set by applying random transformations (e.g., rotations, flips, crops) to existing images, which improves the model's generalization and reduces overfitting. This is the most effective technique among the options because it directly addresses the lack of diverse training examples without changing the model architecture or training hyperparameters in a way that could destabilize learning.

Answer

Increase the learning rate

Answer

Add more layers to the network

Answer

Use a smaller batch size

Question 9

A team stores raw data in S3 and uses a Glue Data Catalog for metadata. They want to allow data scientists to query the data with Amazon Athena using their existing IAM roles. What is the MINIMUM set of permissions required?

Accepted Answer

Grant the IAM role permissions for Athena actions, Glue Data Catalog actions, and S3 read access.. Athena requires read access to S3 for querying data, permissions to the Glue Data Catalog for schema/metadata resolution, and Athena-specific actions to run queries. Write access to S3 is not required for querying, making B the minimum set.

Answer

Grant the IAM role permissions for Athena, Glue, and S3 (read and write).

Answer

Grant the IAM role permissions for Athena and Amazon Redshift Spectrum.

Answer

Grant the IAM role permissions for Athena and Amazon Kinesis.

Question 10

A data scientist is performing EDA on a dataset with mixed data types (numerical, categorical, text). The dataset is stored in S3. Which TWO AWS services can be used to directly perform statistical summaries and visualizations without writing custom code?

Accepted Answer

Amazon SageMaker Data Wrangler. Options D and E are correct. Amazon SageMaker Data Wrangler provides a visual interface for data preparation and analysis with built-in transforms and visualizations directly on S3 data. Amazon QuickSight is a BI service that connects to S3 and creates dashboards with statistical summaries and visualizations. Option A (SageMaker Studio) is an IDE for ML development, not a direct analysis service without custom code. Option B (AWS Glue DataBrew) is a data preparation tool but requires some configuration and is not primarily for statistical summaries and visualizations. Option C (Athena) is a SQL query engine for querying data, but does not provide built-in visualizations.

Answer

Amazon SageMaker Studio

Answer

AWS Glue DataBrew

Answer

Amazon Athena