MLS-C01 Modeling Practice Question

A company is building a binary classifier to predict equipment failure. The dataset has 99% negative (no failure) and 1% positive (failure) examples. The data scientist uses a random forest model with default settings. The model achieves 99% accuracy on the test set but fails to identify any actual failures. Which metric should the data scientist use to evaluate the model?

⚠ Common exam trap

The trap here is that candidates see 99% accuracy and assume the model is performing well, failing to recognize that accuracy is a poor metric for imbalanced datasets, and they overlook recall as the metric that reveals the model's inability to detect the minority class.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Recall

Recall (sensitivity) measures the proportion of actual positive cases correctly identified. With 99% negative examples, a model can achieve 99% accuracy by simply predicting 'no failure' for all instances, but this yields 0% recall for the failure class. Since the goal is to detect rare failures, recall is the appropriate metric to evaluate the model's ability to find positive cases.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
RMSE
Why it's wrong here
RMSE is for regression, not classification.
✗
R-squared
Why it's wrong here
R-squared is for regression, not classification.
✓
Recall
Why this is correct
Recall measures the proportion of actual positives correctly identified, which is critical for imbalanced data.
✗
Precision
Why it's wrong here
Precision is useful but does not capture the failure to identify positives; recall is more relevant here.

Go deeper

MLS-C01 Modeling Practice Question

Why each option matters

Option-by-option breakdown

Related to this question