AI-900 Practice Question: Describe fundamental principles of machine learning on Azure

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✗

  Overfitting to the training data

  Why it's wrong here

  Overfitting to the training data would show up as very high accuracy on the exact examples used for training, but much lower accuracy on a separate test set. This scenario does not describe training accuracy at all; the 95% figure is explicitly the test set accuracy, and the tuning process repeatedly consulted the test set rather than the training set. The failure mode is adaptation to the test data, not memorization of training labels, so this option misidentifies the mechanism.

• ✗

  Data leakage from the training set to the test set

  Why it's wrong here

  Data leakage typically occurs when test-set information (for example, target labels, or statistics such as mean and variance from the full dataset) is inadvertently used during training, such as when scaling transformers are fitted on training and test data together. Here, the test set is used only to evaluate and select hyperparameters, and no test labels are supplied to the learning algorithm itself. The model's parameters are not trained on test examples, so this is not classic leakage; instead it is a form of selection bias caused by tuning on the test set.

• ✓

  Overfitting to the test set

  Why this is correct

  This is correct. When hyperparameters are tuned by repeatedly evaluating against the same test set, the model selection process implicitly 'fits' the test set just as a gradient descent fits the training set. The model becomes tailored to the random noise and idiosyncrasies of that specific test sample, so its apparent 95% accuracy is optimistically biased and will not replicate on fresh data. This is sometimes called test set overfitting or 'testing hypotheses on the test set,' and it inflates reported performance relative to true generalization.

• ✗

  Underfitting the training data

  Why it's wrong here

  Underfitting is a bias problem where the model is too simplistic to capture the underlying patterns, causing poor performance on both training and test data alike. Since the tuned model reportedly reaches 95% accuracy on the test set, it is clearly not underfitting at that stage. The subsequent failure on new, unseen data is not due to an overly simple model, but to the fact that the test set was used during the tuning loop, so the model was effectively selected to match that particular test set rather than to generalize.

Go deeper

Related to this question