A healthcare organization uses an AI model to recommend treatment plans. The model was trained on data from a single hospital, and now treats patients from multiple demographics. Which ethical concern is most critical?
The model trained on a single hospital's data may not generalize, leading to unfair treatment recommendations for other demographics.
Why this answer
The model was trained on data from a single hospital, which likely has a homogeneous demographic profile. When deployed across multiple demographics, the model may produce biased or unfair predictions for underrepresented groups, making fairness and bias the most critical ethical concern. This directly violates the principle of distributive justice in AI ethics.
Exam trap
The AI0-001 exam often tests the distinction between general ethical principles (like accountability or transparency) and the specific, root-cause ethical violation triggered by the scenario, which here is fairness and bias due to demographic mismatch in training data.
How to eliminate wrong answers
Option A is wrong because accountability for treatment outcomes is a general ethical concern but not the most critical here; the primary issue is that the model's training data lacks demographic diversity, which leads to biased predictions before accountability can even be assessed. Option B is wrong because lack of transparency (black-box nature) is a separate concern; while it can exacerbate bias, the core problem is that the model's training data does not represent the target population, not that the model's decisions are opaque. Option C is wrong because privacy violations in training data are a valid concern but not directly triggered by the scenario; the scenario describes using data from a single hospital, which does not inherently imply privacy breaches, whereas the demographic shift introduces bias.