AIF-C01 Guidelines for Responsible AI Practice Question

A hospital uses an Amazon SageMaker model to predict sepsis risk from electronic health records and displays a risk score to clinicians. An internal review finds that the model was trained on data from a single urban hospital and performs worse for patients from rural clinics. The review board asks the data science team to quantify and document this performance gap across patient subgroups before the model is expanded. Which approach should the team take?

⚠ Common exam trap

The trap here is equating Model Monitor drift detection with bias measurement, when drift only signals distribution change and never computes subgroup disparity metrics.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Run SageMaker Clarify bias detection with the rural or urban clinic attribute as the sensitive facet and review the disparity metrics

SageMaker Clarify bias detection measures disparity across a chosen sensitive facet, so using clinic type as the facet quantifies the urban-versus-rural performance gap. The generated report documents the disparity for the review board. Learning rate tuning, Model Monitor drift detection, and Shadow Tests address optimization, operational drift, and traffic comparison respectively, none of which produce subgroup fairness metrics.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✓
Run SageMaker Clarify bias detection with the rural or urban clinic attribute as the sensitive facet and review the disparity metrics
Why this is correct
SageMaker Clarify bias detection accepts a sensitive attribute, called a facet, and computes metrics such as disparate impact and difference in positive proportions across facet values. Using the clinic type as the facet surfaces the quantified performance gap between urban and rural patients. The resulting report can be documented for the review board and used to decide on mitigation or retraining.
✗
Retrain the model with a larger learning rate to improve overall accuracy on the combined dataset
Why it's wrong here
Adjusting the learning rate changes optimization dynamics but does not measure or address subgroup performance differences. Higher overall accuracy can coexist with poor performance for a specific subgroup, masking the very gap the board wants documented. This approach neither quantifies the disparity nor targets the rural patient population, so it fails the review requirement.
✗
Enable SageMaker Shadow Tests to compare the new model against the current clinical workflow
Why it's wrong here
Shadow Tests route a copy of production traffic to a new model variant to compare performance without affecting live decisions. They measure operational performance under real traffic, not fairness across demographic or geographic subgroups. Shadow testing would not produce the disparity metrics the board requires, so it does not satisfy the request.
✗
Deploy the model with SageMaker Model Monitor to detect data drift after expansion to rural clinics
Why it's wrong here
Model Monitor detects drift in production inputs and outputs relative to a baseline, which is useful after deployment. It does not quantify fairness gaps across patient subgroups before expansion, and drift detection is not a bias metric. The review board is asking for a documented subgroup performance analysis, which Model Monitor does not produce.

Go deeper

AIF-C01 Guidelines for Responsible AI Practice Question

Why each option matters

Option-by-option breakdown

Related to this question