AI-900 Practice Question: Describe fundamental principles of machine learning on Azure
What is 'model evaluation' and what metrics are used for different ML task types?
⚠ Common exam trap
It's easy for candidates to assume accuracy is a universal metric, but the AI-900 exam specifically tests that different ML tasks (classification, regression, detection) require specialized metrics like F1, RMSE, and mAP to properly evaluate model performance.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Different tasks use different metrics: F1 for classification, RMSE for regression, mAP for detection
Model evaluation is the process of assessing how well a trained machine learning model performs on unseen data. Different ML task types require different metrics because they measure distinct aspects of performance: for classification tasks, F1-score balances precision and recall; for regression tasks, RMSE (Root Mean Squared Error) quantifies prediction error in the same units as the target; for object detection tasks, mAP (mean Average Precision) evaluates both localization and classification accuracy. Option B correctly identifies these task-specific metrics.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Accuracy is the only metric needed for all ML task types
Why it's wrong here
Accuracy is a misleading metric for imbalanced classification problems, where a model that always predicts the majority class can achieve high accuracy yet be useless; it is also undefined or inappropriate for regression, detection, or translation. Different task types require tailored metrics such as F1 or AUC for classification, RMSE or R² for regression, and mAP for object detection. Relying solely on accuracy ignores these task-specific considerations and leads to incorrect conclusions about model quality.
- ✓
Different tasks use different metrics: F1 for classification, RMSE for regression, mAP for detection
Why this is correct
Evaluation metrics are inherently task-specific because each machine learning task optimizes a different notion of error. For classification, especially with imbalanced classes, F1 score balances precision and recall, while AUC evaluates ranking capability; for regression, RMSE quantifies prediction error in the original units, alongside R² for variance explained. For object detection, mAP combines localization and classification accuracy across confidence thresholds, making it the standard metric.
- ✗
Model evaluation is only needed before deployment, not after
Why it's wrong here
Model evaluation is not a one-time step; it must continue after deployment because models degrade as real-world data drifts. Pre-deployment evaluation on a held-out test set provides only a static snapshot, whereas production data often shifts in distribution, causing performance to decay over time. Continuous monitoring, periodic re-evaluation, and retraining are required to detect and mitigate this drift.
- ✗
The only reliable evaluation is user feedback after the model is deployed in production
Why it's wrong here
While production user feedback is a valuable signal, it is far from reliable or comprehensive: it is biased toward vocal users, arrives with latency, and cannot capture failure modes that users do not explicitly report. Systematic offline evaluation against a labeled held-out test set before deployment is essential for measuring objective metrics like F1, RMSE, or mAP in a controlled manner. Online A/B testing and telemetry complement, but never replace, rigorous offline evaluation.
Go deeper
Related to this question
Learn chapter
Types of AI Workloads
Key term
Prediction
Prediction is the process of using data and algorithms to forecast future outcomes or identify patterns without explicit programming for each scenario.
Key term
Regression
Regression is a type of machine learning algorithm that predicts a continuous numeric output based on input data, used to model relationships between variables.
About these practice questions
One of 985 original AI-900 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-900 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-900 exam.