AI-900 Practice Question: Describe fundamental principles of machine learning on Azure
A bike-sharing company wants to predict the number of rentals per hour. Their model's predictions are usually close but occasionally have large errors due to unexpected events like sudden rain. They want a metric that heavily penalizes these large errors to ensure the model is not overly confident. Which evaluation metric should they primarily use?
⚠ Common exam trap
A common mix-up: candidates choose MAE because it is simpler and more interpretable, but they miss the explicit requirement to 'heavily penalize large errors,' which only MSE (or RMSE) accomplishes through squaring.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Mean Squared Error (MSE)
Mean Squared Error (MSE) is the correct choice because it squares the residuals, which heavily penalizes large errors. Since the bike-sharing company wants to discourage occasional large prediction errors (e.g., due to sudden rain), MSE’s quadratic penalty ensures that models with even a few large outliers receive a much worse score, forcing the model to avoid overconfidence.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Mean Absolute Error (MAE)
Why it's wrong here
Mean Absolute Error sums the absolute differences between predicted and actual rental counts, so each error contributes linearly and equally to the average. Consequently, one wildly inaccurate prediction is penalized no more than a collection of small errors, which directly contradicts the requirement to heavily avoid large deviations. Although MAE is robust to outliers and interpretable, it is not the right metric when large errors carry disproportionate business cost.
When this WOULD be correct
A bike-sharing company wants a metric that is easy to interpret in the original unit (number of rentals) and is robust to outliers. They would use MAE because it gives the average absolute error without disproportionately penalizing occasional large errors.
- ✓
Mean Squared Error (MSE)
Why this is correct
Mean Squared Error squares each residual before averaging, so a prediction that is off by 10 contributes 100 units to the loss while an error of 2 contributes only 4. This quadratic weighting is exactly what the company wants: large errors are penalized disproportionately, preventing a model that tolerates occasional severe under- or over-prediction. Because MSE is also differentiable, it is a natural loss function for gradient-descent-based regression training.
- ✗
Classification Accuracy
Why it's wrong here
Classification Accuracy computes the fraction of predictions whose labels exactly match the true class, which is meaningless for a continuous rental-count target. Rental counts are numerical values, not discrete categories, so there is no class label to match; using this metric would require artificially binning the counts and would ignore how far off a prediction is from the actual number. Accuracy is therefore fundamentally a classification metric, not a regression evaluation metric.
When this WOULD be correct
For a question like 'Which metric should be used to evaluate a model that predicts whether a bike will be rented (yes/no) in the next hour?', Classification Accuracy would be correct.
- ✗
R-squared
Why it's wrong here
R-squared measures the proportion of variance in observed rental counts explained by the model, producing a unitless value between 0 and 1. It is a relative goodness-of-fit score that depends on the total variance of the data, so it does not directly quantify or control the size of individual large residuals. A model with high R-squared can still have a few badly inaccurate predictions, making it unsuitable for a requirement to aggressively penalize large errors.
When this WOULD be correct
A question asking which metric indicates how well the independent variables explain the variability of the dependent variable in a regression model, e.g., 'Which metric shows the proportion of variance in the target explained by the features?'
Option-by-option analysis
Why each answer is right or wrong
Understanding why wrong answers are wrong — and when they would be correct — is what separates a 750 score from a 900. The AI-900 exam frequently reuses these exact scenarios with slightly different constraints.
✓Mean Squared Error (MSE)Correct answer▾
Why this is correct
Mean Squared Error squares each residual before averaging, so a prediction that is off by 10 contributes 100 units to the loss while an error of 2 contributes only 4. This quadratic weighting is exactly what the company wants: large errors are penalized disproportionately, preventing a model that tolerates occasional severe under- or over-prediction. Because MSE is also differentiable, it is a natural loss function for gradient-descent-based regression training.
✗Mean Absolute Error (MAE)Wrong answer — click to see why▾
Why this is wrong here
MAE does not heavily penalize large errors because it treats all errors linearly, whereas the question requires a metric that penalizes large errors more, which is a key property of MSE.
★ When this WOULD be the correct answer
A bike-sharing company wants a metric that is easy to interpret in the original unit (number of rentals) and is robust to outliers. They would use MAE because it gives the average absolute error without disproportionately penalizing occasional large errors.
Why candidates choose this
Candidates may think MAE is always the best choice for regression because it is intuitive and less sensitive to outliers, but they overlook the specific requirement to heavily penalize large errors.
✗Classification AccuracyWrong answer — click to see why▾
Why this is wrong here
Classification Accuracy is used for classification tasks, not regression. The question asks about predicting a continuous number (rentals per hour), so accuracy is inappropriate.
★ When this WOULD be the correct answer
For a question like 'Which metric should be used to evaluate a model that predicts whether a bike will be rented (yes/no) in the next hour?', Classification Accuracy would be correct.
Why candidates choose this
Candidates may confuse regression with classification or think 'accuracy' applies broadly to any prediction task.
✗R-squaredWrong answer — click to see why▾
Why this is wrong here
R-squared measures the proportion of variance explained by the model, not the penalty for large errors. It does not heavily penalize occasional large errors like MSE does.
★ When this WOULD be the correct answer
A question asking which metric indicates how well the independent variables explain the variability of the dependent variable in a regression model, e.g., 'Which metric shows the proportion of variance in the target explained by the features?'
Why candidates choose this
Candidates may confuse R-squared as a metric for error magnitude because it is commonly used in regression evaluation, but it does not directly penalize large errors.
Analysis generated from the official AI-900blueprint and verified against question context. The “when correct” sections are what AI assistants cite when candidates ask “what’s the difference between these options?”
Go deeper
Related to this question
Learn chapter
Large Language Models (LLMs)
Key term
Model
In IT and AI, a model is a trained mathematical representation that learns patterns from data to make predictions or decisions.
Key term
Prediction
Prediction is the process of using data and algorithms to forecast future outcomes or identify patterns without explicit programming for each scenario.
About these practice questions
This AI-900 question is part of Courseiva's 985-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-900 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-900 exam.