Reduce High Bias in Linear Regression with Polynomial Features on SageMaker
A data scientist is using Amazon SageMaker to train a linear regression model. The dataset has 500 features and 50,000 observations. The model converges but has high bias. Which technique should the data scientist use to reduce bias?
Quick Answer
The answer is to add polynomial features or interaction terms to the feature set. This technique directly addresses high bias in linear regression because a linear model with only raw features cannot capture non-linear relationships in the data, leading to underfitting. By introducing polynomial or interaction terms, the model gains the flexibility to fit curvature and complex patterns, thereby reducing bias without increasing variance significantly. On the AWS Certified Machine Learning Specialty MLS-C01 exam, this question tests your understanding of the bias-variance tradeoff and how feature engineering impacts model performance. A common trap is confusing bias-reduction techniques with variance-reduction techniques: regularization (like Lasso or Ridge) reduces variance, not bias, and adding more data also primarily helps with variance. Remember the memory tip: "Bias is about being too simple, so add complexity; variance is about being too complex, so add simplicity or data."
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Add polynomial features or interaction terms to the feature set.
Adding interaction features or polynomial features allows the linear model to capture non-linear relationships, reducing bias. Option A (L2 regularization/ridge) penalizes large coefficients, which primarily reduces variance, not bias. Option C (decreasing the learning rate) affects the step size during gradient descent, influencing convergence speed and stability, but does not directly reduce bias. Option D (feature selection) removes irrelevant features, which can reduce overfitting (variance) and may increase bias if important features are removed. Option E (increasing the number of training epochs) gives the model more iterations to converge, but if the model already converges, more epochs won't reduce bias; it might reduce underfitting if the model hasn't fully converged, but typically bias is addressed by model complexity.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Apply L2 regularization (Ridge) to penalize large coefficients.
Why it's wrong here
Regularization reduces variance, not bias.
- ✓
Add polynomial features or interaction terms to the feature set.
Why this is correct
Increasing model complexity reduces bias.
- ✗
Decrease the learning rate.
Why it's wrong here
Learning rate does not affect bias.
- ✗
Use feature selection to remove irrelevant features.
Why it's wrong here
Removing features can increase bias.
- ✗
Increase the number of training epochs.
Why it's wrong here
More epochs only ensure convergence, not reduce bias.
Go deeper
Related to this question
About these practice questions
Courseiva writes every MLS-C01 question from scratch — 1,672 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on MLS-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A data scientist is using Amazon SageMaker to train a linear regression model. After training, the scientist notices that the model has a high bias. What is the most likely cause?
medium- A.The training dataset has too many features
- B.The model is too complex and overfits the data
- C.The regularization parameter is too high
- ✓ D.The model is too simple and underfits the data
Why D: High bias indicates that the model is underfitting the training data, meaning it is too simple to capture underlying patterns. Option D correctly identifies this cause. Option A is incorrect because too many features typically lead to high variance (overfitting), not high bias. Option B is incorrect because overfitting is associated with high variance, not high bias. Option C is incorrect because while an excessively high regularization parameter can increase bias, it is less likely than the model being too simple; regularization is designed to prevent overfitting, and its improper tuning is not the most common cause of high bias.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.