L2 Regularization for Variance Reduction

Quick Answer

High variance means the model is fitting the training data's idiosyncrasies too closely, and the standard remedy is to constrain the model's complexity in a way that discourages that overfitting without throwing away useful signal, which is exactly what L2 (Ridge) regularization does. L2 regularization adds a penalty term to the loss function proportional to the sum of the squared coefficients, which pushes the model toward smaller coefficient values overall rather than eliminating any of them outright; in practice this means no single feature can dominate the model's predictions the way it might in an unregularized fit, which directly reduces sensitivity to the specific quirks of the training set. This is why it reduces variance without significantly increasing bias: the penalty shrinks coefficients gradually and proportionally rather than forcing a fundamentally simpler model structure, so the model retains the ability to use all the available features while just being less prone to overreacting to noise in any one of them. The other options don't fit as cleanly: adding more features would increase model complexity and likely worsen variance rather than reduce it, and simply training on fewer examples tends to increase bias and can even increase variance further due to less data to learn stable patterns from. Whenever a model shows high variance and the question specifically asks for a fix that avoids meaningfully raising bias, regularization techniques like L2 are usually the intended answer, since they shrink model complexity smoothly rather than through a blunt structural change.