Handling Multicollinearity in Feature Selection
A data scientist is performing EDA on a dataset with 1,000 features and 10,000 rows. The target variable is binary. After checking for multicollinearity, the scientist finds many pairs of features with correlation > 0.95. Which action should be taken to prepare the data for modeling?
Quick Answer
The correct action is to remove one feature from each highly correlated pair based on domain knowledge or higher correlation with the target. This directly addresses handling multicollinearity in feature selection, because when features exhibit correlation above 0.95, they introduce redundancy that destabilizes coefficient estimates in linear models and inflates variance, making the model less reliable and harder to interpret. On the AWS Certified Machine Learning Specialty MLS-C01 exam, this scenario tests your understanding of practical EDA workflows—specifically that removing redundant features is preferred over dimensionality reduction techniques like PCA when interpretability and alignment with a binary target matter. A common trap is to immediately apply PCA, but that transforms features into uninterpretable components and ignores the target variable’s relationship. Memory tip: “Correlated pairs? Cut the weaker link—keep the one that knows the target best.”
⚠ Common exam trap
The MLS-C01 exam often tests the misconception that PCA is the default solution for multicollinearity, but the trap here is that PCA transforms features into uninterpretable components, whereas removing correlated features directly preserves the original feature space and domain relevance.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
For each highly correlated pair, remove one feature based on domain knowledge or higher correlation with target.
When features are highly correlated (e.g., > 0.95), they introduce multicollinearity, which can destabilize coefficient estimates in linear models and reduce interpretability. Removing one feature from each correlated pair based on domain knowledge or its correlation with the target variable preserves predictive power while reducing redundancy. This approach is more targeted than PCA, which transforms features into uncorrelated components but sacrifices interpretability and may not align with the binary target.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Apply PCA to all features to decorrelate them.
Why it's wrong here
PCA reduces dimensionality but loses interpretability and may not be needed.
- ✗
Standardize all features using StandardScaler.
Why it's wrong here
Scaling does not reduce multicollinearity.
- ✓
For each highly correlated pair, remove one feature based on domain knowledge or higher correlation with target.
Why this is correct
This reduces redundancy while retaining predictive power.
- ✗
Randomly drop half of the correlated features.
Why it's wrong here
Random dropping may remove important features.
Go deeper
Related to this question
About these practice questions
One of 1,672 original MLS-C01 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
Same concept, more angles
2 more ways this is tested on MLS-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A data scientist is performing EDA on a dataset with many features. They suspect some features are redundant due to high pairwise correlations. Which technique can help identify groups of correlated features?
medium- A.Use t-SNE to visualize feature relationships
- B.Apply PCA and examine the loadings
- C.Compute mutual information between each feature and the target
- D.Use chi-square test for each pair
- ✓ E.Create a correlation matrix and visualize with a heatmap
Why E: Create a correlation matrix and visualize with a heatmap. This technique directly shows pairwise correlations between features, making it easy to identify groups of highly correlated (redundant) features. Option A is incorrect: t-SNE is a dimensionality reduction technique for visualization of high-dimensional data, but it does not quantify pairwise correlations between features. Option B is incorrect: PCA reduces dimensionality by creating principal components that are linear combinations of original features; while loadings indicate feature contributions, they do not directly show pairwise correlations between original features. Option C is incorrect: Mutual information measures dependency between features and target, not between features themselves. Option D is incorrect: The chi-square test is used for testing association between categorical variables, not for continuous features or pairwise correlation analysis.
Variation 2. A data scientist is performing EDA on a dataset with 100 features. They want to reduce dimensionality by removing highly correlated features. Which TWO approaches are appropriate? (Choose TWO.)
medium- A.Use feature importance from a random forest to select top features.
- B.Remove features with low variance using VarianceThreshold.
- ✓ C.Compute a correlation matrix and remove one feature from each pair with correlation >0.95.
- ✓ D.Use Principal Component Analysis (PCA) and select components that explain 95% of variance.
- E.Apply L1 regularization (Lasso) during model training to zero out coefficients of correlated features.
Why C: Options C and D are correct. Option C directly addresses dimensionality reduction by removing highly correlated features, which reduces redundancy. Option D uses PCA to create uncorrelated components, effectively reducing dimensionality while preserving variance. Option A is incorrect because feature importance from random forest is used for selecting features predictive of the target, not for removing correlated features per se. Option B is incorrect because VarianceThreshold removes features with low variance, not specifically for correlation. Option E is incorrect because L1 regularization (Lasso) is a modeling technique that zeroes out coefficients during model training, not a method for EDA.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.