MLS-C01 Modeling Practice Question
A company is using Amazon SageMaker to train a model. Which TWO metrics should be used to evaluate a binary classification model?
⚠ Common exam trap
The trap here is that candidates often pick Accuracy (A) as a default metric without considering class imbalance, or confuse regression metrics like MAE (E) with classification evaluation, while perplexity (B) is a distractor from NLP contexts.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
AUC
AUC (Area Under the ROC Curve) is a threshold-independent metric that measures the model's ability to distinguish between positive and negative classes across all classification thresholds. For binary classification in SageMaker, AUC is robust to class imbalance and provides a single scalar value representing overall model performance, making it a standard evaluation metric.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Accuracy
Why it's wrong here
Accuracy can be misleading for imbalanced data.
- ✗
Perplexity
Why it's wrong here
Perplexity is for language models.
- ✓
AUC
Why this is correct
AUC is a standard metric for binary classification.
- ✓
F1 score
Why this is correct
F1 score balances precision and recall.
- ✗
Mean Absolute Error
Why it's wrong here
MAE is for regression.
Go deeper
Related to this question
About these practice questions
Courseiva writes every MLS-C01 question from scratch — 1,672 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.