AI-900 Practice Question: Describe fundamental principles of machine learning on Azure
What is 'training data' vs 'test data' in machine learning?
⚠ Common exam trap
Test-takers frequently confuse the purpose of the split (chronological order or labeling method) with the fundamental principle that test data must remain unseen during training to provide an unbiased performance estimate.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Training data fits the model; test data provides an unbiased estimate of real-world performance
Training data is used to fit the model's parameters (e.g., weights in a neural network or split criteria in a decision tree), while test data is held back and used only after training to evaluate the model's performance on unseen data. This separation provides an unbiased estimate of how the model will generalize to real-world data, which is critical for avoiding overfitting. In Azure Machine Learning, this split is typically managed via the `train_test_split` function or automated in AutoML pipelines.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Training data is collected first; test data is older data from an archive
Why it's wrong here
The defining feature of a test set is not when the data was collected, but that the model has never been exposed to it during parameter optimisation. Training and test data are typically taken from the same period and distribution and then partitioned randomly, so archived older data could actually harm evaluation by introducing distribution shift. The purpose of the test set is to provide an honest read of generalisation, independent of data recency.
- ✓
Training data fits the model; test data provides an unbiased estimate of real-world performance
Why this is correct
During training, the algorithm minimises a loss function and adjusts model parameters to fit the patterns in the training data. The test set is never touched during this fitting step; its labels are only revealed when the model is evaluated, giving an unbiased estimate of how the model will perform on new, real-world inputs. This separation is why test accuracy is trusted as a measure of generalisation rather than memorisation.
- ✗
Training data is labelled by humans; test data is labelled automatically by the model
Why it's wrong here
In supervised learning, both training and test datasets require ground-truth labels to compare the model's predictions against known outcomes. Labels for test data are not produced by the model; doing so would make evaluation circular because the model would simply 'predict' whatever it already outputs. Human labeling or a trusted labelling process is used for both sets; the real distinction is that training labels are used to update parameters, while test labels are held back for final performance measurement.
- ✗
Test data is always larger than training data to ensure reliable evaluation
Why it's wrong here
In standard machine learning practice training data is usually the larger split, because more examples help the model learn the underlying structure of the task. A test set only needs to be large enough to yield statistically stable accuracy metrics, not larger than the training set. Reliable evaluation depends more on random sampling, non-overlap, and avoiding data leakage than on test set size.
Go deeper
Related to this question
Learn chapter
Machine Learning Core Concepts
Key term
Overfitting
Overfitting occurs when a machine learning model learns the training data too well, including its noise and outliers, causing it to perform poorly on new, unseen data.
Key term
Test data
Test data is a set of information used to verify that a software application or system works correctly under various conditions.
About these practice questions
Courseiva writes every AI-900 question from scratch — 985 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-900 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-900 exam.