Evaluation Metrics for Imbalanced Classification in AWS Machine Learning Specialty

• A.Accuracy
• ✓ B.Recall
• C.RMSE
• D.Precision

Why B: Option B is correct because recall (true positive rate) measures the ability to find all positive samples, which is critical for imbalanced datasets where accuracy can be misleading. Option A is wrong because accuracy is high but misleading in imbalanced data. Option C is wrong because RMSE is a regression metric, not suitable for classification. Option D is wrong because precision focuses on the accuracy of positive predictions but does not capture missed positives; recall is more important for detecting all positive cases.

• ✓ A.F1 score
• B.Accuracy
• C.Recall
• D.Precision

Why A: With imbalanced data (95% negative, 5% positive), accuracy is high despite poor positive class performance. The F1 score (harmonic mean of precision and recall) is a better metric because it captures both false positives and false negatives. Here, recall is only 10%, so even if precision is high, F1 score will be low, reflecting poor model quality.

• A.R-squared
• ✓ B.F1 score
• C.Accuracy
• D.RMSE

Why B: The F1 score is the harmonic mean of precision and recall, making it ideal for imbalanced datasets where accuracy is misleading. Since the model achieves 99% accuracy by simply predicting the majority class (negative), it fails to capture positive cases; F1 score penalizes this by balancing false positives and false negatives, providing a more truthful performance measure.

• A.Accuracy
• ✓ B.AUC-ROC
• C.Root Mean Squared Error (RMSE)
• D.R-squared

Why B: Option B is correct because AUC-ROC is robust to class imbalance and measures the trade-off between true positive rate and false positive rate. Option A is wrong because accuracy can be misleading with imbalanced data. Option C is wrong because RMSE is for regression. Option D is wrong because R-squared is for regression.

• ✓ A.Recall
• B.Accuracy
• C.Precision
• D.AUC-ROC

Why A: Option A (Recall) is the correct focus because recall measures the proportion of actual positive cases correctly identified. With only 10% recall, the model is missing most positive cases despite high accuracy due to class imbalance. Improving recall directly addresses the goal of detecting the positive class. Option B (Accuracy) is misleading in imbalanced datasets as it can be high even if the model predicts all negatives. Option C (Precision) measures the proportion of positive predictions that are correct, which may not increase recall. Option D (AUC-ROC) is a global metric that may not reflect improvements in recall specifically.

• A.RMSE
• ✓ B.F1 Score
• C.Accuracy
• D.AUC-ROC

Why B: The F1 Score is the harmonic mean of precision and recall, making it ideal for imbalanced datasets where accuracy is misleading. With 95% accuracy but only 10% recall, the model is simply predicting the majority class (negative) almost always, so F1 Score captures the trade-off between false positives and false negatives better than accuracy or AUC-ROC.

• A.F1 score
• B.Mean squared error
• C.Accuracy
• ✓ D.AUC-ROC

Why D: AUC-ROC is robust to class imbalance because it evaluates the model's ability to discriminate between positive and negative classes across all classification thresholds, rather than relying on a single threshold. In this scenario, the model predicts only the negative class, so its true positive rate is 0 and false positive rate is 0, yielding an AUC-ROC of 0.5 (random performance), which correctly reflects the model's lack of predictive power.

• A.Precision
• ✓ B.Recall
• C.AUC-ROC
• D.F1 score

Why B: Recall measures the proportion of actual positive cases that are correctly identified. In this imbalanced dataset, the model has high accuracy but low recall (only 10% of positives caught), so recall is the key metric to improve. Option A (Precision) is not the primary focus because it measures how many predicted positives are correct, not coverage. Option C (AUC-ROC) evaluates the model's ability to distinguish classes overall, not specifically the recall of the positive class. Option D (F1 score) is the harmonic mean of precision and recall, but since recall is very low, F1 is also low; however, recall directly addresses the problem of missing positives.