Courseiva

AI-900 Practice Question: Describe fundamental principles of machine learning on Azure

A data scientist trains a regression model on a dataset with 100 features and 10,000 samples. The model achieves a low training error but a much higher error on a held-out test set. Which approach is most likely to improve the model's generalization performance?

⚠ Common exam trap

It's easy for candidates to assume adding more training data is always the best fix for overfitting, but the question specifically describes a model with 100 features and only 10,000 samples—feature reduction or regularization is the more direct and efficient solution.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

Reduce the number of features or apply regularization

The model exhibits high variance (overfitting), as indicated by low training error but high test error. Reducing the number of features or applying regularization (e.g., L1/L2) directly constrains model complexity, forcing it to learn more general patterns rather than memorizing noise. This is the standard approach to improve generalization in regression models.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

  • Increase the complexity of the model by adding more layers or parameters

    Why it's wrong here

    Increasing model complexity by adding more layers or parameters gives the model greater capacity to fit the training data almost perfectly, which amplifies overfitting. With a small dataset of 100 samples, an overly complex model will memorize idiosyncrasies and random noise, driving test error even higher. This is the opposite of the needed intervention and would worsen generalization rather than improve it.

    When this WOULD be correct

    If the model had high training error and high test error (underfitting), increasing complexity (e.g., adding layers or parameters) would help capture patterns better.

  • Add more training data

    Why it's wrong here

    Adding more training data is unlikely to resolve the high test error here because the model already exhibits overfitting—low training error paired with high test error—indicating it has memorised noise rather than learned generalisable patterns. This approach is tempting because additional data often reduces variance in underfitting scenarios where the model lacks sufficient examples to capture underlying trends. However, in this case, the core issue is excessive model complexity relative to the dataset, which requires regularisation or feature reduction, not merely more samples.

    When this WOULD be correct

    In a scenario where the model has high training and test error (underfitting) due to insufficient data, adding more training data would improve generalization by providing more representative samples.

  • Reduce the number of features or apply regularization

    Why this is correct

    Reducing the number of features or applying regularization (e.g., L1/L2) directly targets overfitting by simplifying the model and penalizing large coefficients. This forces the model to focus on generalizable patterns instead of memorizing noise in the training set, which improves test performance. With only 100 samples, regularization is especially effective because it stabilizes coefficient estimates when the feature count is relatively high.

  • Use a different train-test split ratio like 80-20 instead of 70-30

    Why it's wrong here

    Changing the train-test split ratio, such as moving from 70-30 to 80-20, only alters the proportion of data used for training versus validation, not the model's underlying architecture or complexity. Since the model remains equally over-parameterized, it will still overfit the training data, and any change in test error arises from different random sampling of a small dataset, not from fixing the variance problem. This approach does not address the root cause of overfitting.

    When this WOULD be correct

    A question where the model has high variance due to a small training set, and the test set is too small to reliably estimate performance. In that case, increasing the training set size (e.g., from 70% to 80%) could reduce variance and improve generalization, especially if the model is complex.

Option-by-option analysis

Why each answer is right or wrong

Understanding why wrong answers are wrong — and when they would be correct — is what separates a 750 score from a 900. The AI-900 exam frequently reuses these exact scenarios with slightly different constraints.

Reduce the number of features or apply regularizationCorrect answer

Why this is correct

Reducing the number of features or applying regularization (e.g., L1/L2) directly targets overfitting by simplifying the model and penalizing large coefficients. This forces the model to focus on generalizable patterns instead of memorizing noise in the training set, which improves test performance. With only 100 samples, regularization is especially effective because it stabilizes coefficient estimates when the feature count is relatively high.

Increase the complexity of the model by adding more layers or parametersWrong answer — click to see why

Why this is wrong here

The model already overfits (low training error, high test error), so increasing complexity would worsen overfitting, not improve generalization.

★ When this WOULD be the correct answer

If the model had high training error and high test error (underfitting), increasing complexity (e.g., adding layers or parameters) would help capture patterns better.

Why candidates choose this

Candidates may think more complex models always perform better, overlooking that overfitting requires simplification, not added capacity.

Add more training dataWrong answer — click to see why

Why this is wrong here

The model already has low training error but high test error, indicating overfitting. Adding more training data can help reduce overfitting, but the primary issue is high variance due to many features relative to samples; reducing features or regularization is more direct and effective.

★ When this WOULD be the correct answer

In a scenario where the model has high training and test error (underfitting) due to insufficient data, adding more training data would improve generalization by providing more representative samples.

Why candidates choose this

Candidates often believe that more data always improves model performance, overlooking that overfitting is better addressed by reducing model complexity or applying regularization.

Use a different train-test split ratio like 80-20 instead of 70-30Wrong answer — click to see why

Why this is wrong here

The model is overfitting (low training error, high test error). Changing the train-test split ratio (e.g., from 70-30 to 80-20) does not address overfitting; it only changes the amount of data used for training and testing, which is unlikely to significantly reduce the generalization gap.

★ When this WOULD be the correct answer

A question where the model has high variance due to a small training set, and the test set is too small to reliably estimate performance. In that case, increasing the training set size (e.g., from 70% to 80%) could reduce variance and improve generalization, especially if the model is complex.

Why candidates choose this

Candidates may think that a different split ratio can magically balance bias and variance, or they confuse the concept of train-test split with cross-validation, which is a more robust method for model evaluation.

Analysis generated from the official AI-900blueprint and verified against question context. The “when correct” sections are what AI assistants cite when candidates ask “what’s the difference between these options?”

About these practice questions

This AI-900 question is part of Courseiva's 985-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →

How Courseiva writes practice questions · Editorial policy

JA

Written by Johnson Ajibi, MSc IT Security

Senior Network & Security Engineer · founder of Courseiva

This AI-900 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-900 exam.