MLA-C01 ML Model Development Practice Question

A data scientist is using SageMaker to train an XGBoost model for a regression problem. After training, they evaluate the model on a test set and get an RMSE of 10 and an R² of 0.85. Which additional metric would give the MOST insight into the model's average prediction error magnitude?

⚠ Common exam trap

The trap is picking a familiar classification metric (AUC, F1, confusion matrix) for a regression problem; always check whether the target is continuous before selecting metrics.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Mean Absolute Error (MAE)

MAE measures the average absolute difference between predicted and actual values in the same units as the target, so it directly answers 'how far off is the model on average?' RMSE of 10 is also in target units but is dominated by large errors due to squaring, so MAE complements it by showing typical error magnitude without outlier amplification.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
AUC
Why it's wrong here
AUC measures ranking separation between classes across thresholds, which regression targets do not have. It is tempting as a familiar performance metric, and would be correct for binary classification, but it cannot express average prediction error magnitude for a continuous target.
✗
Confusion matrix
Why it's wrong here
A confusion matrix tabulates classification counts across predicted versus actual classes, so it cannot quantify continuous prediction error for regression. It is tempting because it summarises classifier performance, and would be correct had the XGBoost model performed binary or multiclass classification instead.
✗
F1 score
Why it's wrong here
F1 score combines precision and recall for classification labels, requiring discrete classes that regression targets lack. It is tempting as a standard model-quality summary, and would be correct for an imbalanced classification task, but it yields no measure of continuous prediction error magnitude.
✓
Mean Absolute Error (MAE)
Why this is correct
MAE reports the average absolute difference between predicted and actual values in the target's own units, directly quantifying typical prediction error magnitude. Unlike RMSE, which squares errors and overweights outliers, MAE gives the straightforward average error the stem requests, complementing the existing R² of 0.85.

Go deeper

MLA-C01 ML Model Development Practice Question

Why each option matters

Option-by-option breakdown

Related to this question