AIF-C01 Guidelines for Responsible AI Practice Question
Which TWO actions should a data scientist take to evaluate fairness of a binary classification model using Amazon SageMaker Clarify? (Choose two.)
⚠ Common exam trap
Many candidates confuse bias detection with data preprocessing or model explainability, leading them to select resampling (B) or SHAP values (C) instead of recognizing that SageMaker Clarify specifically provides pre-training and post-training bias metrics as separate evaluation steps.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use post-training bias metrics like Difference in Positive Proportions
Option A is correct because SageMaker Clarify's post-training bias metrics, such as Difference in Positive Proportions (DPP), compare predicted outcomes across facets (e.g., gender or age groups) to quantify disparate impact in the model's actual predictions, which is essential for evaluating fairness of a binary classifier. Option D is correct because pre-training bias metrics like Class Imbalance (CI) measure skew in the underlying training data's label distribution across facets before any model is trained, helping detect whether the data itself could lead to biased outcomes. Together, these two metric types cover both data-level and model-level fairness assessment as Clarify is designed to do. Option B is not a Clarify fairness evaluation action but a data preprocessing technique, and balancing data does not by itself measure bias. Option C, SHAP values, explains feature importance/attribution for interpretability, not fairness metrics. Option E, data quality monitoring on unlabeled data, addresses data drift/quality issues and does not evaluate model fairness.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Use post-training bias metrics like Difference in Positive Proportions
Why this is correct
Difference in Positive Proportions in Predicted Labels is a post-training bias metric, comparing predicted positive rates across facets. It quantifies disparate impact in the model's actual decisions, satisfying the requirement to evaluate fairness of the deployed binary classifier.
- ✗
Ensure the training dataset is balanced by resampling
Why it's wrong here
Resampling alters class balance in the training set; it does not measure bias against a protected attribute, so no fairness metric is produced. It is tempting because balanced data improves predictive performance, yet Clarify's bias metrics require specifying a sensitive attribute and label, not rebalancing.
- ✗
Generate SHAP values for feature importance
Why it's wrong here
SHAP values explain which features drove individual predictions, addressing interpretability rather than bias measurement. It tempts because feature attribution feels related to fairness. Clarify's bias metrics, such as disparate impact and demographic parity, quantify outcome disparity across groups.
- ✓
Use pre-training bias metrics such as Class Imbalance
Why this is correct
Class Imbalance quantifies the skew in label distribution before training, exposing whether one outcome dominates the dataset. SageMaker Clarify computes this as a pre-training bias metric, satisfying the requirement to assess bias at the data stage rather than only after model fitting.
- ✗
Run a data quality monitoring job on unlabeled data
Why it's wrong here
Data quality monitoring inspects feature drift and missing values in unlabeled data, not bias against a protected group, so it produces no fairness metric. It is tempting because Clarify does offer monitoring jobs, but those serve production data-quality alerting rather than the bias analysis this scenario requires.
Go deeper
Related to this question
About these practice questions
One of 862 original AIF-C01 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AIF-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AIF-C01 exam.