AIF-C01 Guidelines for Responsible AI Practice Question

An insurance company uses an Amazon SageMaker model to set premium discounts. An internal audit finds that the model's error rates are substantially higher for one demographic group than another, even though overall accuracy is strong. The data science team must investigate and quantify this disparity before the model can be re-approved. Which approach should they take first?

⚠ Common exam trap

The trap here is jumping to a fix such as retraining or monitoring before measuring the disparity, when the audit first requires quantified subgroup fairness metrics.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Run a SageMaker Clarify bias analysis that computes subgroup performance metrics such as accuracy difference and disparate impact across the demographic facets

Before any remediation, the team must quantify the disparity by computing fairness metrics broken out by demographic facet. SageMaker Clarify bias analysis produces exactly these subgroup metrics, such as accuracy difference and disparate impact, giving the audit the evidence it needs. Retraining, monitoring, and added model capacity all act without first measuring the gap, so they cannot satisfy the re-approval requirement.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Increase the model's complexity by adding more layers to improve fit on the training data
Why it's wrong here
Adding capacity risks overfitting and typically widens gaps between subgroups because the model learns majority patterns more aggressively. It offers no measurement of the disparity and no explanation for why one group has higher error rates. This change would obscure rather than resolve the fairness problem the audit raised.
✗
Retrain the model on a larger dataset and compare overall accuracy before and after
Why it's wrong here
Adding data may improve aggregate accuracy but does not measure or explain the disparity between demographic groups. Without quantifying subgroup error rates first, the team cannot know whether retraining helped the affected group or merely masked the gap in the overall metric. This action skips the diagnostic step the audit explicitly requires.
✗
Deploy the model with Amazon SageMaker Model Monitor to track data drift in production
Why it's wrong here
Model Monitor detects changes in input data distribution and model quality over time, which is valuable for ongoing operations. It does not, however, decompose current performance by demographic facet or quantify the existing fairness gap the audit found. Deploying monitoring leaves the identified disparity unmeasured and unexplained, so re-approval could not proceed on evidence.
✓
Run a SageMaker Clarify bias analysis that computes subgroup performance metrics such as accuracy difference and disparate impact across the demographic facets
Why this is correct
SageMaker Clarify bias analysis computes pre-training and post-training metrics broken out by configured facets, including accuracy difference, recall difference, and disparate impact. That directly quantifies the higher error rate observed for one demographic group and produces the evidence the audit requires. It is the correct first step because measurement must precede any mitigation decision.

Go deeper

AIF-C01 Guidelines for Responsible AI Practice Question

Why each option matters

Option-by-option breakdown

Related to this question