How to Reduce Overfitting in Regression with Regularization
A data scientist is training a regression model to predict house prices in Azure Machine Learning. The model uses features like square footage, number of bedrooms, and location (zip code). The data scientist notices that the model has a very low error on the training data but a high error on the test data. Which technique should the data scientist apply during model training to reduce overfitting by penalizing large coefficients?
Quick Answer
The answer is to use a regularization algorithm like Lasso (L1). This technique directly reduces overfitting in regression with regularization by adding a penalty equal to the absolute value of the coefficient magnitudes, which shrinks some coefficients to zero and performs automatic feature selection. When a model has very low training error but high test error, it has memorized noise rather than learning general patterns, and penalizing large coefficients forces the model to simplify. On the Microsoft Azure AI Fundamentals AI-900 exam, this scenario tests your understanding of how to combat overfitting in Azure Machine Learning, often appearing as a choice between L1 (Lasso) and L2 (Ridge) regularization. A common trap is confusing L2 regularization, which shrinks coefficients but never to zero, with L1’s ability to eliminate irrelevant features entirely. Memory tip: Lasso “L1” can “Lop off” coefficients to zero, leaving only the most important predictors.
⚠ Common exam trap
Many exam-takers confuse regularization with feature scaling or training duration, not realizing that only regularization directly penalizes large coefficient magnitudes to combat overfitting.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use a regularization algorithm like Lasso (L1).
Lasso (L1) regularization adds a penalty equal to the absolute value of the magnitude of coefficients, which can shrink some coefficients to zero, effectively performing feature selection and reducing overfitting. This directly addresses the problem of large coefficients causing the model to fit noise in the training data, leading to high test error.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Use a smaller test set.
Why it's wrong here
A smaller test set would provide a less reliable evaluation of generalization and does not address overfitting during training.
When this WOULD be correct
If the question asked how to reduce variance in model evaluation due to limited data, using a smaller test set (or cross-validation) might be considered, but typically a larger test set is preferred for stable estimates.
- ✗
Apply feature scaling only.
Why it's wrong here
Feature scaling (e.g., normalization) helps gradient descent converge but does not penalize large coefficients or directly reduce overfitting.
When this WOULD be correct
When training a model like SVM or k-NN that is sensitive to feature magnitudes, and the question asks for a preprocessing step to ensure all features contribute equally to distance calculations.
- ✓
Use a regularization algorithm like Lasso (L1).
Why this is correct
Regularization adds a penalty for large coefficients (L1 shrinkage), which forces some coefficients to zero and reduces model complexity, effectively combating overfitting.
- ✗
Increase the number of training epochs.
Why it's wrong here
More training epochs can cause the model to overfit further by learning the training data more precisely, increasing the gap between training and test error.
When this WOULD be correct
In a scenario where a model is underfitting (high error on both training and test data) and training is stopped too early, increasing epochs can help the model converge to a better solution.
Option-by-option analysis
Why each answer is right or wrong
Understanding why wrong answers are wrong — and when they would be correct — is what separates a 750 score from a 900. The AI-900 exam frequently reuses these exact scenarios with slightly different constraints.
✓Use a regularization algorithm like Lasso (L1).Correct answer▾
Why this is correct
Regularization adds a penalty for large coefficients (L1 shrinkage), which forces some coefficients to zero and reduces model complexity, effectively combating overfitting.
✗Use a smaller test set.Wrong answer — click to see why▾
Why this is wrong here
Using a smaller test set does not address overfitting; it reduces the reliability of error estimates and may hide overfitting, but does not penalize large coefficients.
★ When this WOULD be the correct answer
If the question asked how to reduce variance in model evaluation due to limited data, using a smaller test set (or cross-validation) might be considered, but typically a larger test set is preferred for stable estimates.
Why candidates choose this
Candidates may think that a smaller test set will make the test error appear lower, confusing evaluation strategy with regularization.
✗Apply feature scaling only.Wrong answer — click to see why▾
Why this is wrong here
Feature scaling alone does not penalize large coefficients; it only normalizes feature ranges. Overfitting caused by large coefficients requires regularization, not scaling.
★ When this WOULD be the correct answer
When training a model like SVM or k-NN that is sensitive to feature magnitudes, and the question asks for a preprocessing step to ensure all features contribute equally to distance calculations.
Why candidates choose this
Candidates may confuse feature scaling with regularization, thinking that scaling coefficients indirectly reduces their magnitude, or they may know scaling is important but misapply it to overfitting.
✗Increase the number of training epochs.Wrong answer — click to see why▾
Why this is wrong here
Increasing the number of training epochs does not penalize large coefficients; it can actually worsen overfitting by allowing the model to fit the training data even more closely.
★ When this WOULD be the correct answer
In a scenario where a model is underfitting (high error on both training and test data) and training is stopped too early, increasing epochs can help the model converge to a better solution.
Why candidates choose this
Candidates may think that more training always improves performance, confusing the concept of convergence with regularization, and not realizing that overfitting is exacerbated by excessive training.
Analysis generated from the official AI-900blueprint and verified against question context. The “when correct” sections are what AI assistants cite when candidates ask “what’s the difference between these options?”
Go deeper
Related to this question
Learn chapter
Machine Learning Core Concepts
Key term
Regression
Regression is a type of machine learning algorithm that predicts a continuous numeric output based on input data, used to model relationships between variables.
Key term
Training data
Training data is a set of examples used to teach a machine learning model how to make predictions or decisions.
About these practice questions
One of 985 original AI-900 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
Same concept, more angles
2 more ways this is tested on AI-900
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A data scientist is training a regression model to predict house prices. The model performs near perfectly on the training data but poorly on a held-out test set. The scientist suspects the model is memorizing the training data instead of learning general patterns. Which technique is most appropriate to directly address this issue?
hard- A.Increase the size of the training dataset
- B.Increase the complexity of the model (e.g., add more features)
- ✓ C.Apply L2 regularization to the model
- D.Switch to a different regression algorithm
Why C: L2 regularization (also known as Ridge regularization) directly addresses overfitting by adding a penalty term proportional to the square of the model weights to the loss function. This discourages the model from assigning excessively large coefficients to features, forcing it to learn simpler, more general patterns rather than memorizing noise in the training data.
Variation 2. A data scientist trains a regression model to predict house prices using features like square footage, number of bedrooms, and location. The model achieves very high accuracy on the training data but performs poorly on a held-out test set. Which technique should the data scientist apply to reduce overfitting?
medium- A.Increase the number of features
- B.Decrease the training data size
- ✓ C.Use regularization
- D.Increase the number of training epochs
Why C: Regularization (Option C) is the correct technique to reduce overfitting because it adds a penalty term to the loss function (e.g., L1 or L2 regularization), which discourages the model from learning overly complex patterns that fit noise in the training data. This helps the model generalize better to unseen data, such as the held-out test set, by constraining the magnitude of feature weights.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-900 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-900 exam.