PCA for Multicollinearity: Dimensionality Reduction
A data scientist is exploring a dataset with 500 features and 10,000 samples. The data scientist computes the pairwise correlation matrix and finds that many features have correlations above 0.9. The data scientist wants to reduce the dataset to 50 features while preserving as much variance as possible. Which technique should be used?
Quick Answer
Principal Component Analysis (PCA) is the correct technique because it directly addresses dimensionality reduction for multicollinearity by transforming correlated features into a set of uncorrelated principal components, ordered by the variance they explain. When a dataset has 500 features with many pairwise correlations above 0.9, PCA’s orthogonal linear projection captures the maximum possible variance in a lower-dimensional space, so keeping the top 50 components preserves the most information while eliminating redundancy. On the AWS Certified Machine Learning Specialty MLS-C01 exam, this scenario tests your understanding that PCA is the go-to method when the goal is variance preservation under high multicollinearity, not feature selection or interpretability—a common trap is choosing Lasso or feature elimination, which discard features rather than transform them. Remember the memory tip: “PCA prioritizes variance, not variables”—it recombines correlated features into fewer, uncorrelated axes, making it ideal for this exact use case.
⚠ Common exam trap
The MLS-C01 exam often tests the distinction between unsupervised variance-preserving techniques (PCA) and supervised or visualization-specific techniques (LDA, t-SNE), leading candidates to mistakenly choose LDA for dimensionality reduction without recognizing its supervised nature and dimension limit.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Apply Principal Component Analysis (PCA) and keep the top 50 principal components.
Principal Component Analysis (PCA) is the correct technique because it performs an orthogonal linear transformation that projects the original 500 features into a new coordinate system where the axes (principal components) are ordered by the variance they capture. By keeping the top 50 principal components, the data scientist retains the maximum possible variance in the reduced 50-dimensional space, directly addressing the goal of preserving variance while handling high multicollinearity.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Remove all but one feature from each group of highly correlated features.
Why it's wrong here
This approach may discard information that is not redundant.
- ✓
Apply Principal Component Analysis (PCA) and keep the top 50 principal components.
Why this is correct
PCA finds orthogonal directions of maximum variance and can reduce dimensionality effectively.
- ✗
Use Linear Discriminant Analysis (LDA) to project to 50 dimensions.
Why it's wrong here
LDA is supervised and requires a target variable; it may not be appropriate if the goal is unsupervised dimensionality reduction.
- ✗
Use t-Distributed Stochastic Neighbor Embedding (t-SNE) to reduce to 50 dimensions.
Why it's wrong here
t-SNE is non-linear and stochastic; it is not suitable for reducing to a specific number of components for modeling.
Go deeper
Related to this question
About these practice questions
This MLS-C01 question is part of Courseiva's 1,672-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on MLS-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A data scientist is exploring a dataset of customer transactions. The dataset has 1 million rows and 50 columns. The target variable is a binary flag indicating whether a customer churned. The data scientist runs a correlation matrix on all numerical features and finds that two features have a correlation coefficient of 0.98. Which action should be taken to improve model performance?
medium- A.Create an interaction term between the two features.
- ✓ B.Remove one of the two highly correlated features from the dataset.
- C.Increase the regularization parameter (e.g., lambda) in the model.
- D.Apply mean-centering to both features to reduce correlation.
Why B: Two features with a correlation coefficient of 0.98 are nearly perfectly multicollinear. This inflates the variance of coefficient estimates in linear models, making them unstable and reducing interpretability. Removing one of the highly correlated features is a standard dimensionality reduction technique that mitigates multicollinearity without significant information loss, as the remaining feature captures almost the same variance.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.