AI-900 Practice Question: Describe fundamental principles of machine learning on Azure
What is model monitoring in Azure Machine Learning and why is it important?
⚠ Common exam trap
Candidates often confuse operational metrics (like API call count or GPU usage) with model-specific performance monitoring, leading them to pick options that describe infrastructure monitoring rather than model quality tracking.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Tracking model prediction quality and data distribution changes in production to detect degradation
Model monitoring in Azure Machine Learning is the continuous tracking of a deployed model's performance in production, focusing on prediction quality (e.g., accuracy, precision, recall) and data distribution shifts (data drift) to detect degradation over time. This is critical because models can become stale as real-world data evolves, leading to poor business decisions or compliance failures. Azure ML's Model Data Collector and monitoring dashboards automatically capture input data and predictions, alerting data scientists when drift or performance drops below defined thresholds.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Checking how many API calls the model endpoint receives per hour
Why it's wrong here
Counting API calls per hour measures endpoint consumption, which is operational usage monitoring for capacity planning, cost tracking, and SLA validation. It provides no signal about whether the model's predictions remain accurate or whether the underlying data distribution has shifted. Model monitoring, by contrast, evaluates prediction quality (e.g., error rates or business metrics) and compares live feature distributions to training baselines to flag degradation.
- ✓
Tracking model prediction quality and data distribution changes in production to detect degradation
Why this is correct
This is the core of model monitoring: systematically observing a deployed model's prediction performance and the statistical nature of its inputs over time to detect degradation. It involves tracking quality metrics against ground truth when available and running distributional comparisons (such as PSI or KS tests) to identify data drift and concept drift. When degradation is detected, the model can be flagged for retraining or investigation, ensuring continued reliability in production.
- ✗
Monitoring the GPU memory usage during model training
Why it's wrong here
Monitoring GPU memory during training is infrastructure resource telemetry, typically used to optimize hardware utilization, avoid out-of-memory failures, and tune batch sizes or parallelism. This is compute monitoring for the training phase, not model monitoring. It says nothing about how well the model generalizes after deployment, and it cannot catch issues like changing customer behavior or feature drift that erode prediction accuracy in live use.
- ✗
Reviewing model architecture choices for optimization
Why it's wrong here
Reviewing model architecture—such as layer counts, ensemble composition, or regularization—is a design-time activity aimed at improving model capacity or interpretability before deployment. It is a static, one-time assessment during the modeling cycle. Model monitoring, however, is an ongoing production process that observes real-world inference data continuously, watching for subtle shifts in input distributions or prediction reliability that cannot be foreseen during design.
Go deeper
Related to this question
Learn chapter
Machine Learning Core Concepts
Key term
Machine learning
Machine learning is a branch of artificial intelligence where computers learn patterns from data to make decisions or predictions without being explicitly programmed for every task.
Key term
Azure Machine Learning
Azure Machine Learning is a cloud service for building, training, and deploying machine learning models at scale.
About these practice questions
Courseiva writes every AI-900 question from scratch — 985 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-900 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-900 exam.